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Abstract 


Effective  and  efficient  DoD  acquisition  programs  require  the  analysis  of  a 
wide  range  of  materiel  alternatives.  Diversity  among  alternatives,  difficulties  in 
selecting  metrics  and  measuring  performance,  and  other  factors  make  the  Analysis 
of  Alternatives  (AoA)  difficult.  The  benefits  of  alternatives  should  be  included  in  the 
AoA,  but  cost  estimates  dominate  most  AoA  processes.  Incorporating  benefits  into 
AoA  is  particularly  difficult  because  of  the  intangible  nature  of  many  important 
benefits.  The  current  work  addresses  the  need  to  improve  the  use  of  benefits  in  AoA 
by  building  a  system  dynamics  model  of  a  military  operation  and  integrating  it  with 
the  Knowledge  Value  Added  (KVA)  methodology.  The  synergies  may  be  able  to 
significantly  improve  the  accuracy  of  KVA  estimates  in  the  AoA  process.  A  notional 
mobile  weapon  system  was  modeled  and  calibrated  to  reflect  four  weaponized 
Unmanned  Aerial  Vehicles  (UAV).  Modeling  a  hypothetical  AoA  for  upgrading  one  of 
the  UAV  indicated  that  there  were  potentially  significant  synergies  that  could 
increase  the  number  of  alternatives  that  could  be  analyzed,  establishing  common 
units  of  benefit  estimates  for  an  AoA,  improved  reliability  of  an  AoA,  and  improved 
justification  of  AoA  results.  These  can  improve  alternative  selection,  thereby 
improving  final  materiel  effectiveness,  thereby  improving  the  DoD  acquisition 
processes. 

Keywords:  DoD  acquisition  programs,  Alternative  Diversity,  Analysis  of 
Alternatives  (AoA),  Knowledge  Value  Added  (KVA),  Unmanned  Aerial  Vehicles 
(UAV),  alternative  selection 
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Introduction 


The  U.S.  defense  acquisition  process  is  initiated  by  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS),  which  is  one  of  the  three  major 
decision  support  systems  used  in  the  DoD  to  interconnect  and  arrive  at  a  new 
warfighting  capability.  The  JCIDS  formulates  force  requirements  with  a  “top  down” 
approach  that  serves  as  both  a  Joint  Force  integrative  process  and  one  that  can  also 
hierarchically  decompose  the  complexities  of  the  battle  spaces  and  their  critical 
mission  elements.  It  must  also  be  aligned  with  the  Planning,  Programming, 
Budgeting,  and  Execution  System  (PPBES)  funding  process  as  a  way  to  descend 
from  the  strategic  to  the  tactical  in  acquisition  programs  and  budgets.  Guidance  from 
the  current  version  of  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 

3170. Olg  (2009,  p.  a-2  (8))  states, 

When  a  materiel  solution  is  required  by  an  approved  ICD,  the  milestone 
decision  authority  (MDA)  determines  the  scope  of  the  subsequent  analysis  of 
alternatives  (AoA),  the  appropriate  entrance  milestone,  and  designates  the 
lead  component(s)  in  a  Materiel  Development  Decision  (MDD).  The  purpose 
of  the  Materiel  Solution  Analysis  phase  is  to  assess  potential  materiel 
solutions  and  to  satisfy  the  entrance  criteria  for  the  next  program  milestone  as 
designated  by  the  MDA.  If  the  next  phase  per  the  MDA  is  Milestone  (MS)  A, 
then  the  ICD  along  with  the  results  of  the  AoA  form  the  basis  for  the  MS  A 
decision. 

JCIDS  uses  Capability-based  Assessments  (CBA)  to  validate  capability  gaps; 
to  discover  solutions  such  as  those  addressed  by  nonmateriel-type  changes  to 
doctrine,  organization,  training,  leadership  and  education,  personnel,  or  facilities;  or 
to  pursue  materiel  solutions.  Essential  to  CBA  subprocesses  is  the  knowledge  of 
various  functional  warfighting  Joint  Capability  Areas  and  how  those  communities 
operate  within  a  joint  paradigm.  Functional  Capability  Boards  are  organized  around 
top-tier  functional  areas  such  as  Force  Application,  Logistics,  etc. 

Once  needs  are  specifically  derived  in  an  area,  and  it  is  ascertained  that  they 
can  only  be  addressed  by  new  materiel,  the  acquisition  community  is  still  often  left 
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with  either  a  variety  of  system  types  or  technical  approaches  within  a  particular 
system  type  to  fully  address  that  capability  need.  An  in-depth  Analysis  of 
Alternatives  (AoA)  helps  sponsors  and  program  managers  compare  options. 
Examples  include  manned  or  unmanned  aircraft  versus  a  missile,  chemical  energy 
versus  kinetic  energy  kill  mechanisms,  etc.  In  the  past,  these  have  also  been  called 
Cost  and  Operational  Effectiveness  Analyses  and  cost-effectiveness  analyses — all 
variants  of  a  business  case  analysis. 

The  focus  of  the  current  work  is  on  improving  the  Analysis  of  Alternatives 
(AoA)  process  that  is  used  to  make  these  major  acquisition  decisions.  We  will 
demonstrate  the  use  of  a  system  dynamics  modeling  approach  that  incorporates 
common  units  of  benefits  parameters  using  the  KVA  methodology  and  the  potential 
improvements  that  might  result  in  an  AoA. 
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Problem  Description 


In  typical  weapon  system  acquisition  programs,  there  is  a  point  at  which  an 
AoA  is  conducted  to  select  the  most  viable  and  cost-effective  materiel  solution  so 
that  it  may  be  pursued  into  advanced  development  and  production.  Often  a  selection 
for  advanced  technical  development  among  competitive  system  prototypes  is 
needed,  a  practice  that  has  recently  become  official  DoD  acquisition  policy,  but  is 
not  a  new  idea  (USD[AT&L],  2007). 

System  concepts  are  further  clarified  during  the  Materiel  Solutions  Analysis 
phase  and  the  accompanying  AoA  process.  As  part  of  this  process,  programs  move 
toward  several  kinds  of  evaluative  cost  comparisons  that  formulate  costs  estimates 
across  a  notional  lifecycle.  In  these  early  stages,  programs  use  analogous  and 
parametric  cost-estimation  techniques.  Parameters  of  system  performance  and  key 
system  characteristics  are  selected  from  technical  and  operational  inputs.  Usually 
several  Key  Performance  Parameters  included  in  the  Initial  Capability  Document  are 
used  in  the  AoA  to  quantify  points  of  differentiation. 

Simulations  help  address  uncertainty  across  likely  operational  contingencies 
(Ford  &  Dillard,  2008,  2009a,  2009b).  Balancing  of  programmatic  and  operational 
risks  should  be  accounted  for,  but  costs  predominate  most  analyses  because,  for 
major  weapon  systems,  they  can  be  huge.  Not  only  must  research  and  development 
costs  be  considered  but  also  production  costs  and,  beyond  that,  all  of  the  operating 
costs  of  spares,  diagnostics,  maintenance,  tools,  training  manuals,  etc.,  must  be 
estimated.  The  GAO  Cost  Estimation  Guide  (2009)  states, 

For  example,  10  U.S.C.  §  2434  requires  an  independent  cost  estimate  before 
a  major  defense  acquisition  program  can  advance  into  system  development 
and  demonstration  or  production  and  deployment.  The  statute  specifies  that 
the  full  life-cycle  cost — all  costs  of  development,  procurement,  military 
construction,  and  operations  and  support,  without  regard  to  funding  source  or 
management  control — must  be  provided  to  the  decision  maker  for 
consideration.  In  other  cases,  a  program  manager  might  want  initially  to 
address  development  and  procurement,  with  estimates  of  operations  and 
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support  to  follow.  However,  if  an  estimate  is  to  support  the  comparative  AoA, 
all  cost  elements  of  each  alternative  should  be  estimated  to  make  each 
alternative’s  cost  transparent  in  relation  to  the  others,  (p.  47) 

While  the  emphasis  is  clearly  on  cost  in  these  stages,  operational 
effectiveness  must  also  be  considered  because  that  is  where  benefits  are  realized. 
Current  guidance  does  not  provide  a  method  for  estimating  benefits  in  common 
units.  Some  feel  that  the  emphasis  is  disproportionately  on  cost  without  enough 
emphasis  on  benefits.  The  GAO’s  Cost  Estimation  Guide  (2009)  offers  the  following: 

AOA  compares  the  operational  effectiveness,  suitability,  and  LCCE  of 
alternatives  that  appear  to  satisfy  established  capability  needs.  Its  major 
components  are  a  CEA  and  cost  analysis.  AOAs  try  to  identify  the  most 
promising  of  several  conceptual  alternatives;  analysis  and  conclusions  are 
typically  used  to  justify  initiating  an  acquisition  program.  An  AOA  also  looks  at 
mission  threat  and  dependencies  on  other  programs.  When  an  AOA  cannot 
quantify  benefits,  a  CEA  is  more  appropriate.  A  CEA  is  conducted  whenever 
it  is  unnecessary  or  impractical  to  consider  the  dollar  value  of  benefits,  as 
when  various  alternatives  have  the  same  annual  monetary  benefits.  Both  the 
AOA  and  CEA  should  address  each  alternative’s  advantages,  disadvantages, 
associated  risks,  and  uncertainties  and  how  they  might  influence  the 
comparison  [emphasis  added],  (p.  35) 

The  references  here  to  benefits  are  primarily  in  the  form  of  cost  avoidance  or 
cost  savings.  Clearly,  these  are  not  equivalent  to  normal  estimates  of  benefits  in  the 
business  world  where  revenue  is  the  primary  indicator  of  benefit  and  is  not  derived 
from  the  denominator  or  cost  side  of  the  equation.  Monetizing  benefits  as  some  form 
of  cost  savings  or  avoidance  leads  to  a  slippery  slope  where  the  only  indicator  of 
value,  or  numerator  in  a  productivity  equation  such  as  return  on  investment  (ROI),  is 
a  derivative  of  the  denominator,  i.e. ,  cost.  Such  predilections  inevitably  lead  to  the 
lowest  cost  alternatives,  which  may  not  provide  the  highest  benefits. 
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Research  Focus 


Benefits,  in  common  units,  should  be  included  in  AoA  to  enable  higher  fidelity 
comparisons  among  alternatives  on  the  basis  of  value  and  not  just  cost.  But  how  can 
sponsors  and  program  managers  best  valuate  very  real  and  important  but  intangible 
benefits  such  as  combat  effectiveness,  survivability,  or  national  security?  Lacking  a 
credible  ability  to  quantify  such  subjective  or  intangible  benefits  of  the  capabilities  of 
a  system  type  (or  technical  alternative)  is  a  serious  omission  in  any  rigorous 
Analysis  of  Alternatives.  The  experience  of  Colonel  Dillard,  one  of  the  authors  of  this 
report,  includes  several  recent  examples  that  illustrate  the  need  for  more  than  a 
conventional  cost  effectiveness  analysis  to  defend  a  program  requirement  or  a 
system  parameter  of  technical  capability.  Often  a  particular  system  parameter  of 
capability  (e.g.,  weight,  C-130  transportability,  vertical  take-off  and  landing)  becomes 
a  metric  of  program  life  and  death,  but  with  notably  sparse  articulation  of  empirical 
benefit  to  the  customer  or  end-user. 
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The  Case  of  the  Javelin  Anti-Tank  Weapon  System 


The  Javelin  anti-tank  weapon  system  was,  when  it  was  conceptualized, 
merely  named  after  its  requirement  as  the  Advanced  Anti-Armor  Weapon  System- 
Medium  (AAWS-M).  In  1987-1989,  the  U.S.  Army  tested  three  competing 
technologies  to  fulfill  the  operational  need  for  a  one-man-portable  anti-armor  weapon 
system  in  the  medium  range  (1,000-2,000  meter)  category  and  to  replace  its  aging 
and  ineffective  DRAGON  weapon  system.  Principally,  the  weapon  was  to  have  the 
ability  to  defeat  current  and  projected  threat  armored  vehicles  (including  tanks);  have 
a  maximum  range  of  at  least  2,000  meters;  weigh  no  more  than  20.5  kg  (with  under 
15.5  kg  being  desirable);  have  the  ability  to  be  fired  from  enclosed  spaces;  and  have 
the  ability  to  engage  armored  vehicles  under  cover  or  in  hull  defilade.  The  U.S. 
Marine  Corps  agreed  to  these  requirements,  promising  to  pay  for  production  items, 
but  not  to  fund  research  and  development. 

In  August  of  1986,  “Proof  of  Principle”  contracts  of  $30  million  each  were 
awarded  to  three  competing  contractor  teams,  spanning  a  27-month  period  (the 
phase  we  now  call  “Technology  Development”)  to  develop  the  technologies  and 
conduct  a  “fly-off”  missile  competition.  Each  offered  the  needed  capability  solutions 
with  differing  technologies.  Ford  Aerospace  teamed  with  its  partner  Loral  Systems, 
offering  a  laser  beam-riding  missile.  Hughes  Aircraft  teamed  with  Boeing  to  offer  a 
fiber-optic  guided  missile.  Texas  Instruments  teamed  with  Martin-Marietta,  offering 
an  imaging  infra-red  (I2R)  or  forward  looking  infra-red  (FUR)  missile  system.  Each 
candidate  system  also  offered  some  specific  operational  advantages  and 
disadvantages  that  were  almost  impossible  to  quantify  in  terms  of  cost: 

The  Ford/Loral  Laser  Beam  Rider  required  an  exposed  gunner  and  a 
man-in-loop  throughout  its  rapid  flight.  It  was  the  cheapest  at  an 
estimated  $90,000  “cost  per  kill,”  a  figure  that  was  comprised  not  only 
of  average  unit  production  cost  estimates  but  also  of  reliability  and 
accuracy  estimates.  It  was  fairly  effective  in  terms  of  potential  combat 
utility,  with  diminishing  probability-of-hit  at  increasing  ranges. 
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•  The  Hughes/Boeing  fiber-optic  guided  prototype  enabled  an 
unexposed  gunner  (once  launched)  and  also  required  a  man-in-loop 
throughout  its  slower  flight.  It  was  costlier,  but  less  affected  by  range 
accuracy  with  its  automatic  lock-on  and  guidance  in  its  terminal  stage 
of  flight — and  it  even  offered  target  switching.  It  was  also  more  gunner 
-training  (learning)  intensive,  but  could  attack  targets  from  above 
where  their  armor  was  thinnest. 

•  The  FUR  prototype  offered  completely  autonomous  “fire-and-forget” 
flight  to  target  after  launch  and  was  perceived  as  both  costliest  and 
technologically  riskiest.  It  would  have  been  the  easiest  to  train  soldiers 
to  use  and  would  have  been  effective  to  maximum  ranges  by  means  of 
its  target  acquisition  sensor  and  guidance  packages.  It  was  an 
outgrowth  of  a  1980  initiative  by  the  Defense  Advanced  Research 
Project  Agency  (DARPA)  called  Tank  Breaker  that  also  used  “top 
attack”  as  a  more  effective  means  of  armored  target  defeat. 

1988  was  a  busy  year  for  the  AAWS-M  industry  contractors  as  well  as  for  the 
government  acquirers  and  program  sponsors.  All  three  candidate  teams  finally 
began  building  and  flight-testing  their  missile  prototypes.  They  were  also  submitting 
their  bids  to  the  government’s  Request  for  Proposal  for  the  upcoming  advanced 
development  phase.  On  the  government  side,  acquirers  were  evaluating  these  bids 
and  preparing  to  award  the  36-month  Engineering  and  Manufacturing  Development 
(EMD)  phase  contract,  while  sponsors  were  completing  a  Cost  and  Operational 
Effectiveness  Analysis  (COEA)  of  the  three  candidate  AAWS-M  materiel  solutions. 
Each  of  the  teams  enjoyed  generally  successful  missile  flight  test  outcomes  as  the 
Proof  of  Principle  phase  ended.  Each  flew  over  a  dozen  missiles  and  achieved  a 
target  hit  rate  of  over  60%. 

The  Laser  Beam  Rider  candidate  emerged  as  the  winner  of  the  COEA, 
presumably  from  weighted  cost/efficiency  factors.  But  in  a  strange  twist,  the  Source 
Selection  Evaluation  Board  (SSEB),  which  was  deliberating  concurrently,  instead 
chose  the  FUR  candidate,  presumably  because  of  a  bias  toward  “fire-and-forget.”  As 
part  of  a  typical  capability  formulation  process,  technical  constraints  are  deliberately 
avoided  in  requirements  documents  to  allow  and  encourage  a  maximum  range  of 
alternative  solutions  to  the  need  or  capability  deficiency.  While  time  of  flight  and 
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gunner  survivability  were  not  stated  requirements  in  the  AAWS-M  Joint  Required 
Operational  Capability  document  per  se,  fire-and-forget  nevertheless  translated  into 
greatly  enhanced  gunner  survivability  and  overwhelmingly  appealed  to  user 
representatives  (and  government  developers). 

The  EMD  contract  was  awarded  in  June  1989  to  the  Joint  Venture  team  of 
Texas  Instruments  and  Martin-Marietta.  However,  about  18  months  into  this 
program,  serious  technical  problems  doubled  the  expected  cost  of  development  and 
added  about  18  more  months  to  the  originally  planned  36  months  to  complete.  This 
constituted  a  Nunn-McCurdy  breach  of  cost  and  schedule  thresholds,  with  requisite 
congressional  notifications  and  formal  rebaselining  taking  the  better  part  of  the  next 
year  to  accomplish.  Various  technical  issues  plagued  the  program  at  this  point,  with 
system  weight  being  perhaps  chief  among  them.  User  representatives  convened  a 
Joint  Requirements  Overview  Council  (JROC)  to  reevaluate  the  maximum  weight 
requirement  of  45  pounds  and  increased  the  program  threshold  to  49.5  pounds. 
Clearly,  the  Army  and  Marine  Corps  communities  wanted  the  emergent  system  and 
its  planned  capabilities.  But  that  didn’t  resolve  all  of  AAWS-M’s  issues. 

During  the  months  that  the  program  teetered  on  the  brink  of  termination  for  its 
technical  and  business  issues,  the  director  of  the  Office  of  the  Secretary  of  Defense 
Office  of  Program  Analysis  and  Evaluation  (DPAE),  as  a  principle  member  of  the 
DAB,  took  the  program  to  task,  stating  that  if  the  FUR  version  could  not  be  shown 
able  to  achieve  the  same  $90,000  cost  per  kill  as  had  been  estimated  for  the  Laser 
Beam-Rider,  then  the  program  should  be  terminated  and  restarted,  changing 
technologies  and  pursuing  the  less  risky  laser-guided  version.  The  principal  cost 
driver  of  the  FUR  technology  that  enabled  fire-and-forget  was  a  64x64  matrix  (of 
heat  detectors/pixels)  focal  plane  array  (FPA)  to  be  manufactured  by  one  of  the  Joint 
Venture  partners.  These  tiny  microchips  would  comprise  almost  14%  of  the 
estimated  average  unit  production  cost  (UPC)  of  the  entire  missile.  The  ability  of  one 
of  the  few  producers  in  the  world  to  manufacture  them  with  economically  sufficient 
yield — and  to  achieve  their  rigorous  performance  specifications  for  sensitivity — 
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seemed,  for  a  while,  to  hold  the  fate  of  the  entire  program.  Intense  scrutiny  of 
projected  yields  and  production  costs  of  these  critical  components  would  determine 
whether  the  program  was  feasible  from  this  aspect  alone,  some  believed.  But  the 
answer  was  somewhat  ambiguous,  with  roughly  $12,000  being  the  target  for 
average  UPC,  given  a  planned  buy  quantity  of  about  70,000.  The  cost  of  the  FPAs 
wasn’t  the  only  problem  with  them.  But  it  turned  out  that  their  benefits  could  be 
described  in  a  fairly  tangible  way. 

The  AAWS-M  FPA  specifications  were  derived  from  a  scenario-based  target 
list  of  potential  threat  vehicles  in  different  environments  of  atmospheric  temperature, 
humidity,  obscuration,  etc.  When  the  user  community  saw  that  early  developmental 
AAWS-M  focal  plane  arrays  were  not  meeting  the  full  specifications,  they  convened 
another  JROC  to  allow  stepped,  incremental  achievement  of  target  defeat  scenarios 
over  time — something  we  would  now  refer  to  as  evolutionary  growth.  They  stratified 
performance  in  terms  of  levels  A,  B,  and  C  to  convey  degrees  of  target  defeat 
capability  in  focal  plane  arrays.  This  was  a  very  unusual  move  by  sponsors  -  to 
dissect  a  requirement  to  accommodate  the  pace  of  technological  achievement.1  This 
provided  a  qualitative  assessment  of  what  was  achievable  and  satisfactory  for 
system  performance.  Once  again,  the  communities  that  needed  AAWS-M’s 
capabilities  were  trying  to  ease  the  path  forward. 

Fortunately,  independent  program  evaluation  teams  also  reported  that  FUR 
technology  was  progressing  and  would  be  achievable  within  a  rebaselined  program. 
This  joint  position,  along  with  wider  program  advocacy,  curtailed  the  technical  and 
business  arguments  and  the  fire-and-forget  Javelin  was  allowed  to  proceed.  An 
additional  and  more  capable  provider  of  FPAs  was  brought  in  and  accelerated  as  a 
second  source  for  this  critical  component.  After  difficult  advanced  development 
program  challenges,  AAWS-M  eventually  became  the  Javelin — and  is  known  today 


1  This  is  perhaps  not  unlike  today’s  emergence  of  an  Apple  iPhone®  being  followed  soon  after  by 
release  of  a  3G-  and  4G-capable  iPhone®. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-10- 


as  one  of  our  most  successful  combat  systems.  (In  the  end,  Soldiers  and  Marines 
never  had  to  accept  B-  and  C-level  FPA  performance  because  the  full-capable  FPA 
technology  did,  in  fact,  emerge  in  time  for  fielding.  And  system  weight  has  been  held 
just  below  49.5  pounds  throughout  its  many  years  of  production.) 

There  are  many  business  and  public  policy  lessons  to  be  learned  from  the 
Javelin  program.  Within  its  long  saga  from  initial  concept  to  modern-day  deployment 
and  combat  use  are  illustrations  of  requirements  capture,  early  prototyping, 
technology  readiness,  modeling  and  simulation,  economic  forces  of  competition, 
acquisition  strategy,  decision  bureaucracy,  product  discovery,  economies  of  scale, 
etc.  Perhaps  the  best  lesson  learned  from  the  case  presented  here  about  analyzing 
alternatives  is  that  a  single,  unstated,  qualitative  factor  of  performance  (gunner 
survivability)  ultimately  drove  the  choice.  Javelin  had  a  requirements  document  with 
many  pages  of  quantifiable  requirements  stated  as  measures  of  performance  and 
effectiveness.  But  the  parameter  of  system  technology  that  promised  the  most  of 
what  was  impossible  to  quantify  became  the  overriding  factor  in  the  selection  of 
alternatives.  A  magazine  advertisement  purchased  by  the  Joint  Venture  shortly  after 
their  EMD  contract  win  said  it  eloquently:  “Fire  &  Forget  AAWS-M:  The  Gunner 
Wins.”  The  failure  of  the  Javelin  program  to  move  to  the  final  solution  faster  and 
more  directly  was  due  in  large  part  to  the  insufficient  articulation  of  benefits  as  part 
of  the  Analysis  of  Alternatives  process. 

Research  Question 

As  illustrated  by  the  Javelin  program,  there  is  a  basic  need  for  the  use  of  a 
common  units-of-benefit  estimate  in  the  Analysis  of  Alternatives  process.  This 
should  lead  to  including  common  units-of-benefit  estimates  as  well  as  costs  in  the 
acquisition  AoAs.  The  problem  is  to  develop  a  means  to  do  this  more  effectively, 
given  the  nebulous  nature  of  so  many  of  the  critical  benefits  of  weapon  systems. 
How  can  such  a  method  be  consistently  applied  to  many  alternatives  across  a  wide 
range  of  operational  conditions?  The  current  research  examines  how  KVA  can  be 
integrated  with  system  dynamics  modeling  to  generate  defensible  common  units  of 
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benefit  estimates  that  will  improve  the  rigor  of  the  AoA  process  and,  thereby, 
improve  acquisition  processes. 

The  goals  of  the  current  work  are  as  follows: 

■  Examine  how  military  operations  systems  dynamics  (SD)  simulations 
can  be  combined  with  the  KVA  approach, 

■  Identify  potential  advantages  and  disadvantages  of  integrating  military 
operations  simulations  and  the  KVA  approach, 

■  Investigate  the  potential  of  exploiting  the  benefits  from  the  synergy  of 
SD  and  KVA  to  improve  acquisition  AoA  processes,  and 

■  Identify  and  describe  potential  implications  of  the  integration  on 
acquisition  practice. 

Due  to  the  preliminary  nature  of  this  proof-of-concept  study,  precise 
descriptions  of  system  operations  are  necessary.  The  focus  is  on  the  potential 
usefulness  of  integrating  SD  and  KVA. 
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Introduction  to  Knowledge  Value  Analysis 


In  the  U.S.  Military  context,  the  knowledge  value  added  (KVA)  methodology  is 
a  new  way  of  approaching  the  problems  of  estimating  the  productivity  (e.g.,  in  terms 
of  ROI)  for  military  capabilities  embedded  in  processes  such  as  the  CONOPS  for  a 
weapons  system.  In  the  current  study,  we  posited  several  alternative  CONOPS  for  a 
UAV  system  and  used  system  dynamic  modeling  to  evaluate  their  relative 
productivity.  The  KVA  approach  was  used  to  estimate  the  parameters  based  on  the 
system  dynamic  models  by  providing  the  estimates  of  the  relative  productivity  (i.e. , 
the  ROI2)  of  each  alternative. 

In  a  broader  context,  KVA  also  addresses  the  requirements  of  the  many 
Department  of  Defense  (DoD)  policies  and  directives  previously  reviewed  by 
providing  a  means  to  generate  comparable  value  or  benefit  estimates  for  various 
processes  and  the  technologies  and  people  that  execute  them.  It  does  this  by 
providing  a  common  and  relatively  objective  means  to  estimate  the  value  of  new 
technologies  as  required  in  the  following  literature: 

■  The  Clinger-Cohen  Act  of  1996,  which  mandates  the  assessment  of 
the  cost  benefits  for  information  technology  investments; 

■  The  Government  Accountability  Office’s  Assessing  Risks  and  Returns: 
A  Guide  for  Evaluating  Federal  Agencies’  IT  Investment  Decision- 
Making  (1997),  which  requires  that  IT  investments  apply  ROI 
measures; 

■  DoD  Directive  8115.01,  issued  October  2005,  which  mandated  the  use 
of  performance  metrics  based  on  outputs,  with  ROI  analysis  required 
for  all  current  and  planned  IT  investments;  and 

■  DoD  Risk  Management  Guidance  in  the  Defense  Acquisition 
Guidebook,  which  requires  that  alternatives  to  the  traditional  cost 


2  ROI  is  defined  as  the  revenue  cost/cost,  where  revenue  is  defined  as  the  price  per  common  unit  of 
benefit  using  a  market  comparables  approach.  Given  that  the  price  per  common  unit  is  a  constant, 
precision  in  estimating  the  market  comparable  price,  i.e.,  revenue,  is  not  required. 
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estimation  be  considered  because  legacy  cost  models  tend  not  to 
adequately  address  costs  associated  with  information  systems  or  the 
risks  associated  with  them. 

KVA  is  a  methodology  that  describes  all  organizational  outputs  in  common 
units.  This  provides  a  means  to  compare  the  outputs  of  all  assets  (human,  machine, 
information  technology),  regardless  of  the  aggregated  outputs  produced.  Thus,  it 
provides  insights  about  the  productivity  level  of  processes,  people,  and  systems  in 
terms  of  a  ratio  of  common  units  of  output  produced  by  each  asset  (a  measure  of 
benefits)  divided  by  the  cost  to  produce  the  output  By  capturing  the  value  of 
knowledge  embedded  in  an  organization’s  core  processes,  employees,  and 
technology,  KVA  identifies  the  actual  cost  and  value  of  people,  systems,  or 
processes.  Because  KVA  identifies  every  process  required  to  produce  an  output 
and  the  historical  costs  of  those  processes,  unit  costs  and  unit  values  of  outputs, 
processes,  functions,  or  services  are  calculated.  An  output  is  defined  as  the  end 
result  of  an  organization’s  operations;  it  can  be  a  product  or  service,  as  shown  in 
Figure  1 . 


Figure  1.  Measuring  Output 

For  the  purpose  of  the  systems  dynamics  model  developed  for  this  study,  we 
used  KVA  to  describe  the  outputs  of  all  the  processes  and  subprocesses  in  common 
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units.  This  allowed  us  to  make  their  relative  performance  (e.g.,  productivity,  ROIs) 
comparable.  We  used  KVA  to  measure  the  value  added  by  the  human  capital  assets 
(i.e.,  military  personnel  executing  the  processes)  and  the  system  assets  by 
analyzing  the  performances  of  the  processes.  KVA  provided  a  means  to  set  the 
systems  dynamic  model  parameters  so  that  the  results  would  provide  a  way  to 
compare  the  performance  of  various  approaches  to  the  system  problem. 

By  capturing  the  value  of  knowledge  embedded  in  systems  and  in  use  in 
operators  of  the  processes,  KVA  identified  the  productivity  of  the  system-process 
alternatives.  Because  KVA  identified  every  process  output  required  to  produce  the 
final  aggregated  output,  the  common  unit  costs  and  the  common  unit  values  were 
estimated.  This  allowed  for  the  benchmarking  of  various  systems  and  the  processes 
they  supported  with  any  other  similar  processes  across  the  military. 

The  KVA  methodology  has  been  applied  in  over  80  projects  within  the  DoD, 
from  flight  scheduling  applications  to  ship  maintenance  and  from  modernization 
processes  to  the  current  project  analyzing  several  alternative  approaches  to  the 
system  alternatives  problem.  In  general,  the  KVA  methodology  was  used  for  this 
study  because  it  could  do  the  following: 


Compare  alternative  approaches  modeled  with  a  systems  dynamics 
model  in  terms  of  their  relative  productivity; 

Allocate  value  and  costs  to  common  units  of  output; 

Measure  value  added  by  the  system  alternatives  based  on  the  outputs 
each  produced;  and 

Relate  outputs  to  the  cost  of  producing  those  outputs  in  common  units. 


KVA  quantifies  value  in  two  key  productivity  metrics:  Return  on  Knowledge 
(ROK)  and  Return  on  Investment  (ROI).  Calculations  of  these  key  metrics  are  shown 
in  Table  1 
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Table  1.  KVA  Metrics 


Metric 

Description 

Type 

Calculation 

Return  on  Knowledge  (ROK) 

Basic  productivity, 
cash-flow  ratio 

Function  or 
process  level 
performance  ratio 

Outputs-benefits  in 
common  units/cost  to 
produce  the  output 

Return  on  Investment  (ROI) 

Same  as  ROI  at  the 
sub-corporate  or 
process  level 

Traditional 
investment 
finance  ratio 

(Revenue-investment 
cost)/investment  cost 

Based  on  the  tenets  of  complexity  theory,  KVA  assumes  that  both  humans 
and  technology  in  organizations  add  value  by  taking  inputs  and  changing  them  into 
outputs  (measured  in  common  units  of  complexity)  through  core  processes.  The 
amount  of  change  an  asset  within  a  process  produces  can  be  described  as  a 
measure  of  value  or  benefit.  The  additional  assumptions  in  KVA  include  the 
following: 


Describes  all  process  outputs  in  common  units  (e.g.,  using  a 
knowledge  metaphor  for  the  descriptive  language  in  terms  of  the  time  it 
takes  an  average  employee  to  learn  how  to  produce  the  outputs)  and 
allows  historical  value  and  cost  data  to  be  assigned  to  those  processes 
historically. 

All  outputs  can  be  described  in  terms  of  the  time  required  for  a  single 
point-of-reference  learner  to  learn  to  produce  them. 

Learning  Time,  a  surrogate  for  procedural  knowledge  required  to 
produce  process  outputs,  is  measured  in  common  units  of  time. 
Consequently,  units  of  learning  time  are  proportionate  to  common  units 
of  output. 

Common  units  of  output  that  make  it  possible  to  compare  all  outputs  in 
terms  of  cost  per  unit  as  well  as  in  terms  of  value  (e.g.,  price)  per  unit 
because  value  (e.g.,  revenue)  can  now  be  assigned  at  the 
suborganizational  level. 

Assigns  cost  and  revenue  streams  to  suborganizational  outputs,  after 
which  normal  accounting  and  financial  performance  and  profitability 
metrics  can  be  applied  (Housel  &  Kanevsky,  1995;  Pavlou,  Housel, 
Rodgers,  &  Jansen,  2005;  Rodgers  &  Housel,  2006). 
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Describing  processes  in  common  units  also  permits,  but  does  not  require, 
market  comparable  data  to  be  generated,  which  is  particularly  important  for 
nonprofits  like  the  U.S.  Military.  Using  a  market  comparables  approach,  data  from 
the  commercial  sector  can  be  used  to  estimate  price  per  common  unit,  allowing  for 
revenue  estimates  of  process  outputs  for  nonprofits.  This  also  provides  a  common 
units  basis  to  define  benefit  streams  regardless  of  the  process  analyzed. 

KVA  differs  from  other  nonprofit  ROI  models  because  it  can  allow  for  revenue 
estimates,  enabling  the  use  of  traditional  accounting,  financial  performance,  and 
profitability  measures  at  the  suborganizational  level.  KVA  can  rank  processes  or 
process  alternatives  by  their  relative  ROIs.  This  assists  decision-makers  in 
identifying  how  much  of  the  various  processes  or  process  alternatives  add  value. 

In  KVA,  value  is  quantified  in  two  key  metrics:  Return  on  Knowledge  (ROK: 
revenue/cost)  and  ROI  (revenue-investment  cost/investment  cost).  The  raw  data 
from  a  KVA  analysis  can  become  the  input  into  the  ROI  models  and  various 
forecasting  techniques  such  as  real  options  analysis,  portfolio  optimization,  and 
Monte  Carlo  simulation.  By  tracking  the  historical  volatility  of  price  and  cost  per  unit 
as  well  as  ROI,  it  is  possible  to  establish  risk  (as  compared  to  uncertainty) 
distributions,  which  is  important  for  accurately  estimating  the  forecasted  values  for 
portfolio  optimization  and  real  options  analysis. 

The  KVA  method  has  been  applied  to  numerous  military  core  processes 
across  the  Services.  The  KVA  research  has  more  recently  provided  a  means  for 
simplifying  the  input  values  for  portfolio  optimization  and  real  options  analysis  for 
DoD  processes  and  process  assets.  In  the  current  work,  KVA  primarily  provides  a 
methodology  for  comparing  the  relative  values  of  processes  within  military 
operations. 
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Introduction  to  System  Dynamics 


The  system  dynamics  methodology  applies  a  control  theory  perspective  to  the 
design  and  management  of  complex  human  systems.  System  dynamics  combines 
servo-mechanism  thinking  with  computer  simulation  to  analyze  systems.  It  is  one  of 
several  established  and  successful  approaches  to  systems  analysis  and  design 
(Flood  &  Jackson,  1991;  Jackson,  2003;  Lane  &  Jackson,  1995).  Forrester  (1961 ) 
developed  the  methodology's  philosophy,  and  Sterman  (2000)  specified  the 
modeling  process  with  examples  and  described  numerous  applications.  The 
methodology  has  been  extensively  used  for  this  purpose,  including  studying 
development  projects.  The  system  dynamics  perspective  focuses  on  how  the 
internal  structure  of  a  system  impacts  system  and  managerial  behavior  and,  thereby, 
performance  over  time.  The  approach  is  unique  in  its  integrated  use  of  stocks  and 
flows,  causal  feedback,  and  time  delays  to  model  and  explain  processes,  resources, 
information,  and  management  policies.  Stocks  represent  accumulations  or  backlogs 
of  work,  people,  information,  or  other  portions  of  the  system  that  change  over  time. 
Flows  represent  the  movement  of  those  commodities  into,  between,  and  out  of 
stocks.  The  methodology’s  ability  to  model  many  diverse  system  components  (e.g., 
work,  people,  money,  value),  processes  (e.g.,  design,  technology  development, 
production,  operations,  quality  assurance),  and  managerial  decision-making  and 
actions  (e.g.,  forecasting  and  resource  allocation)  makes  system  dynamics  useful  for 
modeling  and  investigating  military  operations,  the  design  of  materiel,  and 
acquisition. 

When  applied  to  acquisition  programs,  system  dynamics  has  focused  on  how 
performance  evolves  in  response  to  interactions  among  development  strategy  (e.g., 
evolutionary  development  versus  traditional),  managerial  decision-making  (e.g., 
scope  developed  in  specific  blocks),  and  development  processes  (e.g., 
concurrence).  System  dynamics  is  appropriate  for  modeling  acquisition  because  of 
its  ability  to  explicitly  model  critical  aspects  of  development  projects.  System 
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dynamics  models  of  development  projects  are  purposefully  simple  relative  to  actual 
practice  in  order  to  expose  the  relationships  between  causal  structures  and  the 
behavior  and  performance  that  they  create.  Therefore,  although  many  processes 
and  features  of  system  design  and  participants  interact  to  determine  performance, 
only  those  that  describe  features  related  to  the  topic  of  study  are  included.  The 
importance  of  deleted  features  can  be  tested  when  system  dynamics  is  used  to  test 
the  ability  of  the  model  structure  to  explain  system  behavior  and  performance. 

System  dynamics  has  been  successfully  applied  to  a  variety  of  development 
and  project  management  issues,  including  rework  (Cooper,  1993a,  1993b,  1993c; 
Cooper  &  Mullen,  1993),  the  prediction  and  discovery  of  failures  in  project  fast-track 
implementation  (Ford  &  Sterman,  2003b),  poor  schedule  performance  (Abdel- 
Hamid,  1988),  tipping  point  structures  in  projects  (Taylor  &  Ford,  2006,  2008), 
contingency  management  (Ford,  2002),  resource  allocation  (Joglekar&  Ford,  2005; 
Lee,  Ford,  &  Joglekar,  2007),  and  the  impacts  of  changes  (Cooper,  1980),  and 
concealing  rework  requirements  on  project  performance  (Ford  &  Sterman,  2003a). 
See  Lyneis  and  Ford  (2007)  for  a  review  of  the  application  of  system  dynamics  to 
projects  and  project  management. 

System  dynamics  has  also  been  applied  to  military  systems,  including 
planning  and  strategy  (Bakken  &  Vamraak,  2003;  Duczynski,  2000;  McLucas,  Lyell, 
et  al. ,  2006;  Melhuish  et  al. ,  2009),  workforce  management  (Bell  &  Liphard,  1978), 
technology  (Bakken,  2004),  command  and  control  (Bakken  &  Gilljam,  2003;  Bakken 
et  al.,  2004),  operations  (Bakken  et  al.,  2004;  Coyle  &  Gardiner,  1991),  logistics 
(Watts  &  Wolstenholme,  1990),  acquisition  (Ford  &  Dillard,  2008,  2009a,  2009b; 
Bartolomei,  2001;  Homer  &  Somers,  1988),  and  large  system  programs  (Cooper, 
1994;  Lyneis,  Cooper,  &  Els,  2001).  Coyle  (1996)  provides  a  survey  of  applications 
of  system  dynamics  to  military  issues. 

Based  on  the  literature  described  above  and  the  authors’  experience  with 
system  dynamics,  there  appears  to  be  an  opportunity  to  exploit  the  capabilities  of  the 
system  dynamics  methodology  to  make  the  knowledge  value  added  approach  more 
accurate. 
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Research  Methodology 


In  the  current  work,  KVA  and  system  dynamics  were  integrated  to  test  their 
ability  to  improve  the  precision  of  AoAs  in  acquisition  programs.  We  first  developed 
and  tested  a  generic  structure  of  a  mobile  weapon  system  process  using  the  system 
dynamics  methodology.  Then  we  operationalized  KVA  value  and  cost  estimates  in 
the  system  dynamics  model.  The  model  was  calibrated  to  reflect  four  extant 
weaponized  Unmanned  Aerial  Vehicles  (UAVs).  We  used  one  of  those  calibrations 
as  the  basis  for  employing  the  model  in  a  hypothetical  AoA  for  upgrading  the  UAV  to 
address  a  different  type  of  target.  We  analyzed  simulation  results  to  test  the  ability  of 
the  system  dynamics  model  to  estimate  benefits  streams  using  KVA  in  terms  of  the 
relative  value  added  of  the  capabilities  of  the  system. 
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A  Generic  Model  of  Mobile  Weapons  Use 


The  system  dynamics  model  has  three  sectors:  weapons  movement,  target 
evolution,  and  KVA  analysis.  As  we  will  describe,  the  model  structure  simulates  two 
critical  aspects  of  mobile  weapon  system  operations:  (1 )  the  support  and  movement 
of  the  weapon  and  (2)  target  evolution  from  identification  through  confirmation  of 
destruction. 

The  Weapons  Movement  Sector 

The  Weapons  Movement  sector  of  the  model  simulates  the  positions  and 
movements  of  weapons  (e.g.,  individual  UAVs  or  Javelin  gunners).  Figure  2  shows 
the  positions  that  weapons  (generically  called  assets)  can  take  (boxes)  and  the  rates 
of  their  movements  from  one  position  to  another  (arrows  between  boxes).  We 
assumed  that  the  total  number  of  assets  remained  constant,  i.e.,  no  weapons  were 
added  or  lost  during  operations.  This  assumption  can  be  relaxed  when  modeling  a 
specific  asset.  The  movement  of  weapons  is  a  subprocess  of  operating  the  weapon 
system  that  adds  value  and  imbeds  learning  into  tools,  requires  learning  time  for 
operators  to  be  capable  of  performing  and  requires  processing  time  to  accomplish. 
Therefore,  the  completion  of  moving  weapons  to  the  station  and  back  to  the  base  is 
an  output  of  that  subprocess  and  an  input  to  the  KVA  analysis.  The  combination  of 
the  two  movements  “Assets  arrive  at  station  rate”  and  “Assets  arriving  as  base  rate” 
represent  the  accomplishment  of  the  vehicle  movement  subprocess. 
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Avg  time  at  base 


Figure  2.  Positions  and  Movement  of  Weapons  During  Operations 

Each  rate  in  the  weapons  sector  describes  the  average  movement  of  the 
weapons  in  the  accumulation  that  precedes  the  rate.  Each  rate  is  defined  with  the 
number  of  weapons  preparing  for  that  rate  (to  leave  base  or  station)  or  event  (to 
arrive  at  station  or  base)  and  the  average  time  spent  by  a  weapon  in  the  preceding 
accumulation.  For  example,  the  (average)  “Assets  leaving  base  for  station”  rate  is 
equal  to  the  number  of  assets  at  the  base  divided  by  the  average  time  that  a  weapon 
spends  at  the  base  between  trips  to  the  station.  This  formulation  increases  the 
average  departure  rate  with  more  weapons  at  the  base  and  decreases  the  average 
departure  rate  if  weapons  stay  at  the  base  longer.  The  average  time  at  the  base  is 
characteristic  of  particular  assets  and  can  generate  different  behaviors  and 
performances  across  weapons  and  configurations. 
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The  Target  Evolution  Sector 


The  Target  Evolution  sector  of  the  model  simulates  the  development  of 
targets  through  five  subprocesses  of  system  operations: 

1.  Acquire  target — includes  detection,  recognition,  location,  classification 
(identification),  and  confirmation  (Global  Security,  2010). 

2.  Fire  support  coordination — allocates  targets  to  weapons  by  a  group  of 
people  that  have  access  to  information  about  the  battlefield  situation 
and  about  doctrine,  major  systems,  significant  capabilities  and 
limitations,  and  often  their  tactics,  techniques,  and  procedures  (TTP) 
(Williams,  2001). 

3.  Fire  mission  development — prepares  specific  instructions  and  target 
information  for  transmission  to  the  weapons  team  and  to  the  weapon 
(e.g.,  target  location  coordinates). 

4.  Engage  target — weapons  operators  (e.g.,  pilots  for  UAV)  maneuver  the 
weapon  within  striking  distance  of  the  target,  enter  the  target 
coordinates,  and  launch  munitions. 

5.  Battlefield  assessment — often  the  same  asset  as  was  used  for  target 
acquisition  is  used  to  evaluate  the  success  of  engagement  in 
destroying  the  target. 

In  the  model,  targets  evolve  through  these  stages  in  an  “aging  chain” 
structure  of  sequential  accumulations  (backlogs  +  work  in  progress,  referred  to  here 
as  backlogs)  and  (sub)processes  that  drain  those  backlogs  and  contribute  to  the 
backlog  of  the  next  downstream  subprocess.  Figure  3  shows  the  conditions  of 
targets  (boxes)  and  the  rates  of  their  movements  from  one  condition  to  another 
(arrows  between  boxes)  due  to  subprocesses.  The  movements  “Acquire  target 
completion  rate,”  “Fire  support  coordination  to  asset,”  “Fire  mission  completion  rate,” 
“Engage  target,”  and  “Battlefield  assessment  rate”  are  subprocesses  that  add  value, 
imbed  learning  in  tools,  require  learning  time  for  operators  to  be  capable  of 
performing  and  require  processing  time  to  complete.  Therefore,  they  are  each 
outputs  of  those  subprocesses  and  inputs  to  the  KVA  analysis. 
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rate 


Figure  3.  Accumulations  and  Movements  of  Targets  in  Weapon 

System  Operations 

In  addition  to  the  primary  flows  of  targets  through  the  subprocesses,  the 
target  sector  models  three  common  causes  of  mission  failure:  (1)  hitting  the  target 
but  failing  to  destroy  it,  (2)  missing  the  target,  and  (3)  missing  the  target  and  losing 
the  location  information  needed  to  engage  the  target  again  (e.g.,  because  the  target 
moved).  Each  cause  moves  the  target  to  a  different  condition  in  the  target  aging 
chain.  Hitting  the  target  but  failing  to  destroy  it  (e.g.,  a  hardened  target)  requires 
reengagement,  but  often  no  additional  targeting  information.  After  Battlefield 
assessment,  these  targets  are  returned  to  the  Target  engagement  backlog.  Missing 
the  target  (e.g.,  a  small  target)  requires  that  the  fire  mission  be  developed  again  to 
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re-aim  the  weapon  prior  to  reengagement.  Therefore,  these  targets  are  returned  to 
the  Targets  in  fire  mission  development  backlog  after  Battlefield  assessment.  Losing 
the  target  (e.g.,  a  fast-moving  vehicle)  requires  that  the  target  be  reacquired. 
Therefore,  these  targets  are  returned  to  the  Targets  being  acquired  backlog  after 
Battlefield  assessment. 

In  a  manner  similar  to  the  modeling  of  the  movement  of  weapons,  the  rates  in 
the  target  sector  describe  the  average  movement  of  targets  between  backlogs.  The 
primary  rates  in  the  aging  chain  are  defined  by  the  number  of  targets  in  the  backlogs 
of  the  subprocess  and  the  average  time  required  to  perform  the  subprocess.  The 
average  time  required  to  perform  the  subprocess  is  characteristic  of  particular 
subprocesses  (e.g.,  different  engagement  durations  for  different  weapons)  and  can 
generate  different  behavior  and  performance  across  weapons  and  configurations. 
When  two  or  more  flows  drain  a  backlog  (Targets  in  fire  support  coordination  and 
Battlefield  assessment  backlog),  the  total  outflow  is  split  between  the  flows  using  a 
percent  that  leaves  the  stock  through  each  outflow.  The  return  flows  are  each  a 
fraction  of  the  Battlefield  assessment  rate.  Those  fractions  are  based  on  the  ability  of 
the  weapon  to  successfully  destroy,  hit,  and  not  lose  targets.  Therefore,  like  in 
practice,  different  weapon  alternatives  (e.g.,  range,  payload,  dash  speed)  impact 
mission  success.  The  Using  System  Dynamics  and  KVA  to  Improve  Analysis  of 
Alternatives  section  of  this  report  describes  how  these  features  of  the  model  were 
used  to  describe  operational  scenarios  and  weapon  configurations.  The  fractions  of 
the  Battlefield  assessment  rate  that  was  returned  to  the  engagement  backlog  due  to 
being  hit  but  not  destroyed,  was  missed  and  returned  to  the  mission  development 
backlog,  or  was  lost  and  returned  to  the  target  acquisition  backlog  are  described  with 
the  probability  of  destruction  if  the  target  is  hit  with  the  ordinance  (p(kill  if  hit)  or 
p(kill)),  the  probability  of  the  weapon  hitting  the  target  with  ordinance  (p(hit)),  and  the 
probability  of  not  losing  the  target  if  it  is  missed  with  the  ordinance  (p(not  lose)), 
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respectively.3  These  probabilities  are  determined  by  comparing  the  ability  of  a 
weapon  to  successfully  destroy,  hit,  and  not  lose  targets  to  the  characteristics  of  the 
target.  More  specifically,  the  probability  of  kill  is  modeled  with  the  weapon’s  payload 
compared  to  the  lethal  payload  (i.e. ,  ordinance  size  required  to  kill);  probability  of  hit 
is  modeled  with  the  weapon’s  dash  speed  compared  to  the  target’s  speed;  and  the 
probability  of  not  losing  the  target  is  modeled  with  the  weapon’s  range  compared  to 
the  target’s  distance  from  the  base.  Therefore, 

p(kill)  =  fk(Payload  /  Lethal  payload) 

p(hit)  =  fh(Dash  speed  /  Target  speed) 

p(not  lose)  =  fni(Range  /  Target  distance  from  base) 

where, 

p(kill):  probability  of  destruction  if  the  target  is  hit  with  the  ordinance 
p(hit):  probability  of  the  weapon  hitting  the  target  with  ordinance 
p(not  lose):  probability  of  not  losing  the  target  if  it  is  missed  with  the 
ordinance 

The  three  functions  that  estimate  the  probabilities  based  on  the  ratios  are 
assumed  to  be  simple  but  realistic  relations  that  include  the  entire  range  of  possible 
conditions.4  The  function  relating  the  Payload/Lethal  payload  ratio  to  the  probability 
of  kill  is  assumed  to  increase  linearly  from  p(kill)=0  when  the  ratio  is  zero  (i.e.,  no 
payload  prevents  any  chance  of  target  destruction)  to  p(kill)=1 00%  when  the  ratio  is 
greater  than  or  equal  to  1  (i.e.,  if  the  payload  exceeds  the  lethal  payload,  the  target 
is  assumed  to  be  destroyed  if  it  is  hit).  The  function  relating  the  Dash  speed/Target 
speed  ratio  to  the  probability  of  the  weapon  hitting  the  target  with  ordinance 
assumes  that  the  vehicle  will  “chase”  a  moving  target  and  that  the  faster  the  vehicle 
is,  the  closer  it  can  get  to  the  target  before  releasing  ordinance — increasing  the 


3  The  probability  of  not  losing  a  target  is  used  instead  of  the  probability  of  losing  a  target  to  retain  a 
“bigger  is  better”  standard  for  all  three  measures  and,  therefore,  to  facilitate  intuitive  understanding  of 
the  model. 

4  These  functions  can  be  described  more  accurately  with  additional  weapon  testing  information. 
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likelihood  of  hitting  the  target  with  the  ordinance.  However,  there  is  always  some 
possibility  of  missing  a  target,  even  if  the  vehicle  is  faster  than  the  target  and, 
therefore,  close  to  the  target.  The  function  is  assumed  to  have  an  elongated  S  shape 
from  p(hit)=0  when  the  ratio  is  zero  (i.e.,  no  Dash  speed  prevents  hitting  the  target) 
to  p(hit)=90%  when  the  ratio  is  greater  than  or  equal  to  three  (i.e.,  high  likelihood  of 
hit  if  the  weapon  speed  far  exceeds  target  speed.5  The  function  relating  the 
Range/Target  distance  from  base  ratio  to  the  probability  of  not  losing  the  target  if  it  is 
missed  with  the  ordinance  assumes  that  the  vehicle  will  move  toward  the  target  but 
that  the  target  may  also  move,  sometimes  closer  to  the  vehicle  and  sometimes  away 
from  it.  When  the  target  moves  away  from  the  vehicle,  it  may  move  out  of  the 
vehicle’s  range,  causing  the  vehicle  to  lose  the  target.  The  function  is  assumed  to 
have  a  stretched  out  S  shape  from  p(not  lose)=0  when  the  ratio  is  zero  (i.e.,  no 
weapon  range  causes  the  vehicle  to  always  lose  the  target)  to  p(not  lose)=95% 
when  the  ratio  is  greater  than  or  equal  to  1 .8,  reflecting  some  chance  of  losing  the 
target  even  if  it  is  well  within  the  vehicle’s  range. 

The  KVA  Sector 

The  KVA  metrics  were  fully  operationalized  within  the  system  dynamics 
model.  The  KVA  sector  uses  operations  information  from  the  weapons  and  targets 
sectors  of  the  model  and  characteristic  descriptors  of  weapons  to  generate  relative 
value  metrics  for  each  subprocess  (including  weapons  capability  outputs)  of  the  UAV 
operations.  KVA  generates  a  productivity  ratio  that  reflects  output/input.  If 
monetized,  this  ratio  can  be  a  traditional  benefit-cost  ratio  (e.g.,  ROI  if  benefits  are 
monetized  as  a  form  of  revenue  surrogate).  Other  measures  of  benefits  and  costs 
can  also  be  used — as  long  as  there  are  common  units  in  the  numerator  (benefits) 
and  all  the  cost  units  of  all  the  contributors  to  the  denominator  are  the  same  (as  is 
most  often  the  case  because  costs  are  almost  always  monetized) — so  they  can  each 


5  The  assumed  function  relating  the  Dash  Speed/Target  Speed  to  the  probability  of  hitting  the  target  is 
probably  lower  than  current  experience,  but  is  used  to  reflect  the  change  in  targets  described  in  the 
Using  System  Dynamics  and  KVA  to  Improve  Analysis  of  Alternatives  section  of  this  report. 
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be  aggregated.  At  each  point  in  time,  each  subprocess’s  productivity  is  the  benefits  it 
has  generated  divided  by  the  costs  of  generating  those  benefits  (i.e., 
output/input). The  model  includes  both  monetized  and  time-based  KVA  metrics. 

Monetized  KVA  Metrics 

When  monetized,  the  numerator  (benefits)  for  each  subprocess  is  the 
accumulation  across  operations  of  a  fraction  of  the  value  added  by  one  complete 
operation  (e.g.,  in  this  case  it  would  be  the  monetary  value  of  the  destruction  of  a 
target).  The  fraction  allocated  to  each  subprocess  is  directly  proportional  to  the 
Learning  time  required  to  produce  the  outputs  of  the  subprocess,  in  common  units  of 
learning  time.  Learning  time  captures  the  benefits  derived  from  human  processing 
(e.g.,  flying  the  vehicle),  automated  processes  (e.g.,  takeoffs  and  landings),  and 
(importantly)  technologies  integrated  into  the  weapon.  The  denominator  of 
monetized  subprocess  KVA  productivities  is  the  accumulation  across  operations  of 
the  costs  incurred  to  perform  the  subprocess.  How  these  were  modeled  is  described 
next. 


The  benefit  generated  by  a  weapon  system  is  the  destruction  of  targets.  That 
benefit  can  be  monetized  with  an  estimate  of  the  value  (in  monetary  terms)  of  the 
destruction  of  a  typical  or  average  target.  This  value  can  be  estimated  with  the  cost 
that  the  government  would  have  to  pay  a  private  entity  to  perform  the  same  task 
without  the  use  of  the  weapon  system  (e.g.,  using  a  land-based  operation  instead  of 
an  aircraft  or  missile).  The  benefits  generated  by  each  subprocess  are  a  fraction  of 
the  total  benefits  that  the  operation  has  accumulated  so  far.  That  fraction  is  modeled 
as  being  directly  proportional  to  the  learning  times  (reflecting  complexity  as  a 
common  units  measure  of  value  added)  of  the  subprocesses.  Those  total  benefits  of 
a  subprocess  are  the  accumulation  over  time  of  the  product  of  the  subprocess’s 
portion  of  the  benefits  for  a  single  target  and  the  rate  at  which  the  subprocess  is 
being  performed.  Based  on  this,  a  set  of  equations  for  estimating  the  benefits  of  a 
single  subprocess  are  as  follows: 
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Unit  subprocess  benefit  fraction  =  Subprocess  learning  time  / 

Total  of  all  subprocess  learning  times 

Unit  subprocess  benefit  =  Unit  subprocess  benefit  fraction  * 

Unit  benefit  for  entire  process 

Rate  of  subprocess  generating  revenue  =  Subprocess  processing  rate  * 

Unit  subprocess  benefit 

Subprocess  benefits  generated  to  date  =  ^(Rate  of  subprocess  generating 

benefits)  *  dt 


where, 

dt:  timestep,  the  period  over  which  the  argument  is  integrated 

The  denominator  of  KVA  subprocess  productivity  ratios  represents 
subprocess  costs.  The  cost  to  date  at  any  time  is  modeled  as  the  product  of  the  time 
required  to  perform  the  subprocess  and  the  average  hourly  cost  of  performing  the 
subprocess.  The  total  time  spent  performing  a  subprocess  at  any  given  time  is  the 
product  of  the  average  time  required  for  the  subprocess  and  the  number  of 
performances  of  the  subprocess.  Therefore,  a  set  of  equations  for  estimating  the 
costs  of  a  single  subprocess  in  each  time  period  are  as  follows: 

Rate  of  spending  time  on  subprocess  performance  = 

Subprocess  performance  rate  * 
time  required  to  perform  the  subprocess 

Subprocess  work  time  spent  to  date  = 

(Rate  of  spending  time  on  subprocess  performance)  *  dt 

Subprocess  processing  time  generated  to  date  = 

Subprocess  work  time  spent  to  date  *  Hourly  performance  cost 


In  each  time  period,  subprocess  benefits  and  costs  are  combined  into 
subprocess  KVA  productivity  ratios. 

Subprocess  productivity  =  Subprocess  benefits  generated  to  date  / 

Subprocess  processing  time  generated  to  date 


NPS 
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The  calculation  of  monetary  KVA  metrics  has  been  incorporated  into  the 
system  dynamics  model.  However,  the  modelers  decided  that  the  current  model 
would  be  simpler  to  interpret  by  using  the  non-monetized  common  units  of  output  (as 
described  in  terms  of  the  units  of  time  it  would  take  the  average  person  to  learn  how 
to  produce  the  outputs)  as  the  numerator.  This  results  in  a  standard  definition  of 
productivity  (output/input). 

Time-Based  KVA  Metrics 

Calculating  the  learning  time-based  KVA  metrics  applies  the  same  approach 
as  the  monetized  metrics,  but  uses  Learning  Time  to  quantify  benefits  and  Touch 
Time  to  quantify  costs  instead  of  money.  One  of  several  ways  to  quantify  a 
subprocess’s  Learning  Time  is  to  estimate  the  average  time  required  for  a  common 
point-of-reference  learner  to  be  trained  and  become  competent  in  performing  the 
subprocess.  Each  subprocess  is  assigned  a  unit  learning  time  that  reflects  the 
relative  (compared  to  other  subprocesses)  complexity  of  the  subprocess.  Each 
subprocess  is  also  assigned  a  Unit  Touch  Time  that  reflects  the  relative  (compared 
to  other  subprocesses)  effort  required  to  perform  the  subprocess.  These  are 
aggregated  over  many  operations  and  compared  in  order  to  generate  a 
subprocess’s  productivity  ratio.  The  equations  for  modeling  time-based  KVA  metrics 
for  a  subprocess  are  as  follows: 

Subprocess  learning  Time  accumulated  to  date  = 

Y  (Rate  of  subprocess  operation  *  Subprocess  unit  learning  time)  *  dt 

Subprocess  touch  time  accumulated  to  date  = 

Y  (Rate  of  subprocess  operation  *  Subprocess  unit  touch  time  *  dt 

Subprocess  Productivity  =  Subprocess  learning  time  accumulated  to  date  / 

Subprocess  touch  time  accumulated  to  date 
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Learning  Times  and  Touch  Times  are  also  aggregated  across  subprocesses 
in  order  to  estimate  the  productivity  of  the  entire  operation.  This  allows  the 
comparison  of  different  asset  configurations  (alternatives). 
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A  Brief  Introduction  to  Weaponized  Unmanned 
Aerial  Vehicles 


Unmanned  Aerial  Vehicles  (UAVs)  have  a  long  history,  dating  from  their  use 
as  targeting  drones  in  the  1920s  (Jones,  1997).  They  have  evolved  into 
reconnaissance  assets,  valuable  particularly  where  air  space  is  considered  too 
dangerous  for  manned  aircraft.  Other  advantages,  such  as  the  ability  to  remain  on 
station  for  extended  periods  of  time,  reduced  mission  costs,  and  access  to  space 
denied  to  manned  aircraft,  have  been  exploited  in  more  recent  UAVs.  In  the  DoD, 
UAVs  are  divided  into  three  classes: 

■  Close  Range  UAVs  (UAV-CR)  that  operate  within  a  range  of  about  50 
kilometers; 

■  Short  Range  UAVs  (UAV-SR)  with  8-10-hour  flight  durations,  a  range 
of  200  kilometers,  and  the  ability  to  communicate  through  a  datalink; 
and 

■  Endurance  UAVs  (UAV-E)  with  at  least  24-hour  flight  durations  and  the 
ability  to  perform  multiple  missions  simultaneously. 

Weaponized  UAVs  were  developed  and  used  in  the  Vietnam  War.  The  Navy’s 
DASH  UAV  helicopter  carried  two  250-pound  torpedoes  that  were  used  to  destroy 
North  Vietmanese  supply  barges  in  the  Mekong  Delta  waterways  after  detection  by 
the  vehicle’s  television  camera  (Global  Security,  2010).  However,  technical  and 
other  limitations  prevented  the  further  operationalization  of  weaponized  UAVs  for 
several  decades.  Several  weaponized  UAVs  are  now  operational,  including  the  MQ- 
1  Predator,  MQ-9  Reaper,  and  Sky  Warrior,  with  the  last  two  being  decedents  of  the 
Predator.  Other  weaponized  UAVs  are  under  development,  including  the  X-45B 
(also  known  as  the  Unmanned  Combat  Air  System-Navy  or  UCAS-N)  by  Northrop 
Grumman  (Figure  4)  that  will  include  aircraft  carrier  suitability  (Cayas,  2007),  and  is 
being  tested  on  the  USS  Abraham  Lincoln  (Marks,  2010). 
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Figure  4.  An  Artist’s  Rendering  of  an  X-47B 
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Model  Calibration  and  Testing 


The  KVA+SD  model  was  calibrated  to  the  operations  of  four  actual 
weaponized  UAVs.  The  operations  included  the  following  subprocesses: 

1.  acquire  target, 

2.  fire  support  coordination, 

3.  fire  mission  development, 

4.  move  weapons, 

5.  engage  targets,  and 

6.  perform  battlefield  assessments. 

Three  of  the  four  UAVs  are  operational:  Predator,  Sky  Warrior,  and  Reaper. 
The  fourth  UAV  is  the  X-47B.  The  modelers  collected  basic  characteristics  relating  to 
UAV  performance  for  the  four  UAVs  from  publicly  available  sources  (such  as  Global 
Security,  2010).  That  information  included  vehicle  range,  total  mission  time,  time  on 
station,  dash  speed,  and  payload.  The  DoD  has  developed  multiple  versions  of 
some  of  the  vehicle  with  different  characteristics.  In  these  cases,  a  single  version 
was  selected  and  used.  Other  information  was  estimated  for  each  vehicle’s 
operations,  including  learning  and  processing  times  for  each  subprocess. 

Reasonable  assumptions  were  used  in  making  these  estimates;  for  example,  we 
made  the  assumption  that  the  time  required  to  engage  a  target  after  arriving  on 
station  was  inversely  proportional  to  the  vehicle’s  dash  speed  (i.e.,  faster  dash 
speeds  reduced  the  time  required  to  engage).  These  estimates  were  rough  but 
adequate  for  this  proof-of-concept  study,  which  sought  to  determine  if  the  model  was 
capable  of  reflecting  differences  in  characteristics  in  KVA  parameters,  not  whether  it 
was  capable  of  predicting  actual  outcomes. 

The  model  was  tested  using  standard  tests  for  system  dynamics  models 
(Forrester  &  Senge,  1980;  Sterman,  2000),  including  for  structural  similarity  to  the 
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actual  system,  reasonable  behavior  over  a  wide  range  of  input  values,  and  behavior 
similarity  to  actual  systems.  Basing  the  model  structure  on  previously  validated 
models  and  the  literature  improves  the  model’s  structural  similarity  to  actual 
acquisition  projects.  Model  behavior  (e.g.,  simulated  sizes  of  backlogs  for 
subprocesses  and  rates  of  performing  operations)  was  compared  to  typical  behavior 
and  found  to  be  similar.  For  example,  before  operations  started  at  the  beginning  of 
the  operational  scenario  (described  in  what  follows),  the  backlogs  were  empty  and 
no  operations  were  being  performed.  The  appearance  of  targets  increased 
subprocess  backlogs  and  rates  of  operation  as  weapons  left  base  and  subsequently 
arrived  on  station,  acquired  targets,  coordinated  fire,  developed  missions,  engaged 
targets,  and  assessed  the  battlefield.  Figure  5  shows  an  example  of  these  simulated 
backlogs  for  the  Predator  UAV. 


Target  Backlogs 


Targets  being  acquired  :  Predator-Base  Case  -  targets 

Targets  in  fire  support  coordination  :  Predator-Base  Case  -  targets 

Targets  in  fire  mission  development  :  Predator-Base  Case  -  targets 

Target  engagement  backlog  :  Predator-Base  Case  -  targets 

Targets  in  battlefield  assessment  backlog  :  Predator-Base  Case  -  targets 


NOTE:  Target  Backlog  is  measured  in  the  number  of  targets 

Figure  5.  Backlogs  in  the  Target  Evolution  Sector — Predator  UAV 

Base  Case 

In  the  evolution  of  targets,  these  backlogs  and  subprocesses  increase 
sequentially  through  the  series  of  operations.  The  growth  of  operations  and  backlogs 
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slows  as  capacities  adjust  to  demand  (backlog  sizes)  until  the  operations  are  in 
dynamic  equilibrium  conditions,  with  sizes  of  backlogs  and  operations  rates 
remaining  within  a  relatively  narrow  range.  This  represents  “steady  state”  operations 
that  could  be  continued  for  a  significant  period  of  time,  e.g.,  until  damage  to 
weapons  or  maintenance  (not  included  in  the  current  model)  changes  weapon 
availability.  Model  behavior  was  also  tested  with  extreme  input  values  such  as 
perfect  operations  (e.g.,  probability  of  hit=1 00%)  and  a  very  large  versus  very  small 
number  of  weapons  and  targets  as  well  as  with  more  typical  conditions.  Model 
behavior  remained  defensible  across  wide  ranges  of  input  values,  including  extreme 
values.  These  tests  increased  modeler  confidence  that  the  model  generates  realistic 
operational  behavior  patterns  due  to  the  same  causal  relations  found  in  the  type  of 
operations  investigated  (i.e.,  generates  “the  right  behavior  for  the  right  reasons”). 

The  operational  scenario  was  described  with  the  quantity  and  characteristics 
of  the  targets.6  A  stream  of  targets  entered  the  target  acquisition  backlog  at  a  steady 
rate  of  five  targets  per  minute.  The  target  distance  from  the  base  was  assumed  to 
vary  uniformly  from  400-1 ,100  nm.  This  distance  described  targets  including  those 
that  were  closer  to  the  weapon’s  base  than  the  shortest  weapon’s  range  to  targets 
that  were  farther  from  the  base  than  the  longest  weapon’s  range.  The  speed  of  the 
targets  was  assumed  to  vary  uniformly  from  50  to  250  nm.  This  described  targets 
from  those  that  were  almost  immobile  to  targets  that  were  faster  than  the  fastest 
weapon  modeled.  The  payload  required  to  destroy  the  target  if  hit  (i.e.,  lethal 
payload)  was  assumed  to  vary  uniformly  from  400  to  1 ,000  lbs.  This  described 
targets  from  those  that  were  very  soft  to  targets  that  were  very  hardened. 

KVA  productivities  for  the  six  subprocesses  and  the  cumulative  productivity  of 
those  processes  for  the  four  weaponized  UAVs  are  shown  in  Table  2.  Each  KVA 


6  Although  a  single  operational  environment  was  simulated  for  this  research,  multiple  and  different 
environments  can  be  simulated.  Examples  of  characteristics  of  the  operational  scenario  that  can  be 
elaborated  include  dynamic  variation  in  the  entering  target  rate,  distributions  of  target  characteristics, 
and  more  target  characteristics. 
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productivity  ratio  is  described  as  the  integer  resulting  from  dividing  the  benefits  (in 
Learning  Time)  by  the  cost  (in  Touch  time).  For  example,,  for  the  Fire  Mission 
Development  subprocess  for  the  four  UAVs  those  ratios  are  943  (Predator),  3,122 
(Reaper),  1,222  (Sky  Warrior),  and  3,962  (X-47B).  They  represent  the  benefits 
(output)  per  unit  of  cost  (input)  and,  therefore,  can  also  be  interpreted  as  a  measure, 
in  percentages,  of  the  return  on  the  investment.  These  values  remained  constant  in 
the  model  after  steady  state  operations  had  been  established.  As  an  example  of  the 
components  of  the  ratios,  the  Fire  Mission  Development  subprocess  ratio  for  the 
Predator  (943)  is  the  quotient  of  the  accumulated  benefits  (e.g.,  after  five  hours  of 
operations)  of  79,684  learning-time  hours  and  84.5  processing-time  hours.  In  the 
simulated  steady  state  operations,  these  accumulated  learning-time  hours  increased 
at  a  rate  of  301  learning-time  hours  per  minute  (the  product  of  the  estimated  500 
learning-time  hours  per  fire  development  operation  and  an  average  fire  development 
rate  of  0.6  targets  developed  per  minute),  and  the  processing-time  hours  increased 
at  a  rate  of  0.3  hours  per  minute  (the  product  of  the  estimated  30-minute  processing 
time  to  develop  a  fire  mission  and  the  same  average  fire  development  rate  of  0.6 
targets  developed  per  minute).  Transitional  periods  (e.g.,  the  start  or  end  of 
operations)  or  other  nonsteady  state  operations  can  generate  ratios  that  vary  over 
time. 


Table  2.  KVA  Productivity  Ratios 
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44 
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Battlefield  assessment 
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Weapon 
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Note  that,  as  described  in  The  KVA  Sector  section,  these  productivities  are 
ratios  of  accumulated  learning  time  divided  by  accumulated  processing  time. 
Therefore,  they  are  relative  values.  As  expected,  the  three  productivities  for  the 
subprocesses  that  were  not  impacted  by  the  characteristics  of  the  vehicle  (acquire 
targets,  fire  support  coordination,  and  battlefield  assessment)  did  not  change.  These 
subprocesses  were  not  impacted  by  different  vehicles  because  the  subprocess  was 
the  same  for  all  of  these  vehicles.  The  application  of  system  dynamics  and  KVA  to 
the  Analysis  of  Alternatives  of  other  system  alternatives — such  as  improved  logistics 
or  vehicle  technology  used  for  recognizing  and  indentifying  targets — would  generate 
changes  in  these  KVA  productivities.  However,  three  important  subprocesses  that 
impacted  total  product  productivity  (see  the  row  titled  Weapon  in  Table  2)  did  vary 
(fire  mission  development,  move  weapons,  and  engage  targets). 

Some  of  the  ratios  in  Table  2  are  relatively  large  when  compared  to  returns 
on  investment  experienced  in  many  industries,  especially  for  the  Engage  targets 
subprocess.  A  primary  reason  is  that  the  numerator  of  these  ratios  includes  the 
benefits  of  the  technologies  incorporated  into  the  UAV  for  target  engagement 
purposes.  These  technologies  are  extremely  complex,  are  reflected  in  very  large 
learning-time  hours  that  are  accumulated  each  time  a  target  is  engaged,  and, 
therefore,  generate  high  productivity  ratios.  Similarly,  the  denominator  of  these  ratios 
reflects  the  time  required  to  perform  the  subprocess,  e.g.,  engage  a  target  after  it 
has  been  acquired,  coordinate  fire  support,  develop  fire  missions,  and  move  UAVs 
to  stations.  Actual  engagement  times  are  relatively  short  for  these  UAVs,  further 
increasing  the  KVA  productivity  ratios  for  the  engage  target  subprocess.  Differences 
in  learning  times  across  the  UAVs  reflect  their  relative  performance  (e.g.,  the 
automation  of  subprocesses  previously  performed  by  humans).  Technologies  are 
the  primary  causes  of  differences  in  the  ratios  across  the  UAVs  in  Table  2. 

Therefore,  it  is  reasonable  that  the  very  large  benefits  of  the  X-47B,  with  its 
extremely  advanced  technology,  generate  the  largest  ratios.  Improved  estimates  of 
learning  times  and  processing  times  can  increase  the  accuracy  of  these  ratios. 
However,  a  comparison  of  the  KVA  productivity  ratios  for  the  engage  targets 
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subprocess  with  the  ratios  for  the  move  weapons  subprocess,  which  is  simpler 
(lower  numerator)  and  takes  longer  (larger  denominator),  indicates  that  the  rank 
order  of  the  ratios  reflects  the  relative  returns  of  the  different  subprocesses. 

Based  on  these  and  additional  tests,  the  model  is  considered  useful  for  the 
investigation  of  the  integration  of  system  dynamics  and  KVA.  Table  2  indicates  that 
the  X-47B  is  the  most  efficient  (generates  more  benefits  for  given  costs).  The  next 
section  uses  the  model  to  generate  a  deeper  understanding  of  how  subprocesses 
contribute  to  overall  weapon  productivity.  . 
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Using  System  Dynamics  and  KVA  to  Improve 
Analysis  of  Alternatives 


Consider  the  following  hypothetical  example  of  the  use  of  an  integrated 
system  dynamics/KVA  model  to  improve  the  productivity  estimates  supporting  an 
Analysis  of  Alternatives.  Assume  that  a  new  version  of  the  Predator  UAV  is  being 
developed  to  enable  it  to  engage  opposing  UAVs.  Due  to  the  much  higher  speeds 
and  agility  of  UAVs  compared  to  most  land-based  targets,  the  fraction  of  targets 
missed  is  expected  to  be  higher  than  that  currently  experienced  with  the  Predator. 
The  acquisition  program  management  team  has  access  to  some,  albeit  limited, 
resources  (e.g.,  money,  expert  developer  time  until  required  delivery,  technology 
development  capabilities,  approvals)  to  improve  performance.  Different  stakeholders 
value  payload,  dash  speed,  and  range  differently  and  want  the  program 
management  to  recommend  different  improvements.  Therefore,  program 
management  expects  a  rigorous  review  of  its  Analysis  of  Alternatives  process  and 
the  results  that  will  recommend  one  (and  only  one)  of  the  improvements.  As  part  of 
the  justification  of  the  AoA  decision,  stakeholders  of  the  two  solutions  not 
recommended  are  certain  to  require  explanations  of  how  and  how  much  the 
recommended  improvement  will  impact  operational  performance  compared  to  the 
improvements  that  were  not  recommended.  Cost  would,  most  likely,  be  their  primary 
economic  consideration,  as  evidenced  by  the  earlier  case-study  examples.  However, 
our  analysis  will  focus  on  value  compared  to  cost  in  terms  of  the  capabilities  of  the 
systems. 

Many  alternatives  have  been  proposed  and  are  being  considered.  A  few 
examples  are7: 


7  There  are  interdependencies  and  trade-offs  in  these  alternatives,  such  as  needing  to  increase  the 
size  of  the  power  plant  to  maintain  a  given  dash  speed  if  the  size  of  the  fuel  tank  is  increased.  These 
are  ignored  here  for  simplicity.  However,  in  an  application  to  an  actual  program,  developers  would 
describe  specific  sets  of  features  (e.g.,  possible  versions  of  a  vehicle)  for  analysis. 
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Increase  the  size  of  the  power  plant,  which  can  be  used  to  increase  the 
vehicle’s  payload,  dash  speed,  or  a  combination  of  both.  This  requires 
an  increase  in  fuel  capacity  in  order  to  not  reduce  range. 


■  Redesign  the  transmission,  which  will  increase  the  vehicle’s  dash 
speed. 

■  Increase  the  fuel  tank  size,  which  will  increase  the  vehicle’s  range  but 
decrease  its  dash  speed  unless  the  power  plant  is  also  increased. 

■  Reduce  the  time  required  at  base  between  trips  to  station,  which  will 
increase  the  time  that  the  vehicle  is  on  station  and  available  for 
missions. 

Performing  detailed  analyses  of  all  the  possible  alternatives,  such  as  by 
building  and  testing  prototypes  or  very  detailed  simulations,  can  exceed  the 
resources  of  acquisition  programs.  Therefore,  program  managers  may  be  faced  with 
the  challenge  of  reducing  a  long  list  of  potential  alternatives  to  those  that  should 
definitely  be  included  in  the  program,  those  that  should  be  investigated  further  for 
potential  inclusion,  and  those  that  should  be  rejected.  The  integration  of  system 
dynamics  and  KVA  provides  a  timely  and  inexpensive  means  of  evaluating  all 
potential  alternatives  and  reducing  the  long  list  of  potential  alternatives  to  a  short  list 
to  be  pursued  or  investigated  further  based  on  an  objective  and  justifiable  process. 
To  do  this,  the  operation  of  the  system  with  each  potential  alternative  must  first  be 
simulated.  This  simulation  can  provide  insight  into  system  operations  based  on 
different  investment  decisions.  For  example,  Figure  6  shows  the  simulated  evolution 
of  the  backlog  of  targets  waiting  for  engagement  based  on  three  investment  choices: 
an  increased  power  plant  that  doubles  the  allowable  payload  (100%  power  plant 
payload),  a  Predator  without  improvement  (Predator  base  case),  and  a  redesigned 
transmission  that  doubles  the  vehicle’s  dash  speed  (100%  faster  dash  speed). 

These  can  be  used  to  improve  CONOPS  (e.g.,  the  number  of  different  types  of 
assets  allocated). 
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Target  Engagement  Backlog 


Time  (Minute) 

Target  Engagement  Backlog  :  Predator-+100%  Power  plant-payload  \ \ \ \ \ \ \ 

Target  Engagement  Backlog  :  Predator-Base  Case  2222222222 

Target  Engagement  Backlog  :  Predator-- 100%  fester  dash  speed 

Figure  6.  Engagement  Backlog  Evolution  for  Five  Predator 

Investment  Choices 

The  simulations  can  also  be  used  to  calculate  the  KVA  productivity  ratios  for 
the  subprocesses  and  for  the  system  as  a  whole.  Table  3  provides  an  example  of  a 
portion  of  such  an  analysis  for  the  hypothetical  upgrading  of  the  Predator  UAV  using 
the  model  described  in  the  A  Generic  Model  of  Mobile  Weapons  Use  section.  . 
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Table  3.  Predator  UAV  Upgrade  Program 


Fire  mission 
development 
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Move 
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Predator  base  Case 
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0% 

Increase  fuel  capacity  100% 
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Increase  power  plant  100% 
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^  100%  faster  dash  speed 
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O  Increase  power  plant  100% 
for  payload 

943 

50 

10,188 
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2% 

Redesign  transmission  for 
50%  faster  dash  speed 

943 

78 

7,641 

717 

2% 

Reduce  time  at  base  50% 

943 

50 

5,094 

699 

-1% 

Reduce  time  at  base  100% 

943 

52 

5,094 

693 

-2% 

KVA  Productivity  Ratios  for  Analysis  of  Alternatives 

The  KVA  productivity  ratios  are  repetitive  for  some  subprocesses  across 
alternatives.  This  is  partially  because  some  alternatives  do  not  change  the  impact  on 
some  subprocesses  and  partially  because  of  the  limited  number  of  system 
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interactions  incorporated  into  this  proof-of-concept  model.  However,  the  results  from 
using  an  integrated  system  dynamics/KVA  model  are  adequate  to  show  how  more 
accurate  results  might  be  used  in  an  AoA  of  potential  capabilities  upgrades.  Based 
on  the  results  above,  a  program  manager  can  assess  the  relative  value  added  by 
the  eleven  alternatives  (including  no  change,  as  reflected  by  the  Base  Case) 
analyzed.  A  comparison  to  the  base  case  (e.g.,  the  existing  vehicle  in  the  case  of  the 
Predator  upgrade)  provides  an  estimate  of  relative  performance  improvement.  By 
sorting  the  improvements  provided  by  potential  alternatives  in  decreasing  order 
(Table  3),  we  created  a  list  of  alternatives  from  most  attractive  (increase  fuel 
capacity  100%)  to  least  attractive  (reduce  time  at  base  50%).  The  AoA  suggests  that 
if  adequate  resources  are  available,  then  the  alternative  that  improves  the  system 
the  most  is  to  increase  the  fuel  capacity  100%  because  it  improves  the  development 
of  the  fire  missions.  If  inadequate  resources  are  available  to  implement  this 
alternative,  then  the  program  should  attempt  to  increase  the  fuel  capacity  by  50%  for 
similar  reasons.  The  program  manager  can  also  remove  from  consideration  reducing 
the  time  at  the  base  and  a  50%  increase  in  power  plant  capacity  that  would  be  used 
to  increase  dash  speed  because  they  do  not  improve  performance.  Certainly  other 
factors  must  be  incorporated  into  a  complete  AoA  (most  notably  development  costs), 
but  the  results  of  the  KVA  analysis  using  the  system  dynamics  model  provide 
valuable  information  for  making  final  recommendations. 
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Discussion  and  Conclusions 


Summary 

The  Knowledge  Value  Added  (KVA)  approach  to  including  benefits  in  an 
Analysis  of  Alternatives  (AoA)  was  integrated  with  a  system  dynamics  model  of 
weapon  systems  operations  to  investigate  the  potential  of  their  integration  to 
improve  the  accuracy  of  KVA  productivity  ratios  and,  thereby,  AoA.  An  integrated 
model  was  developed  for  a  generic  mobile  weapons  system  and  calibrated  to  four 
existing  weaponized  Unmanned  Aerial  Vehicles  (UAV).  Six  basic  subprocesses  of 
operations  using  the  weapons  were  included  in  the  simulation.  KVA  productivity 
ratios  for  each  subprocess  for  each  UAV  were  calculated,  compared,  and  used  to 
explain  how  the  simulation  and  KVA  approach  work  together  to  generate  quantitative 
assessments  of  the  relative  value  added  of  each  subprocess  and  whole  weapon 
system.  A  hypothetical  upgrade  program  to  one  of  the  UAVs  was  also  simulated  to 
demonstrate  how  the  integrated  model  can  be  used  to  evaluate  alternative  upgrades 
and  justify  AoA  decisions. 

Evaluation  of  Results 

An  Analysis  of  Alternatives  based  on  an  integrated  system  dynamics/KVA 
model  provides  program  management  teams  with  several  kinds  of  valuable 
information: 


Quantified  Measures  of  Improvement  that  include  Benefits: 

Measures  of  subprocesses  and  the  weapon  system  as  a  whole  are 
quantified  using  a  common  set  of  assumptions  and  values  (those 
incorporated  into  the  simulation  model).  Therefore,  differences  in  ratios 
and  the  implied  relative  value  of  different  alternatives  are  due  to  the 
differences  in  the  alternatives  themselves. 

Overall  System  Improvement  Estimates:  The  weapon  (versus  the 
subprocess)  -productivity  ratios  reflect  changes  in  total  product 
operations.  If  adequate  resources  are  available  to  adopt  at  least  one 
alternative,  then  a  list  of  alternatives  ranked  by  overall  system 
improvement  (e.g.,  Table  3)  can  be  used  to  “triage”  alternatives  into 
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those  that  should  definitely  be  pursued,  those  that  require  more 
investigation  before  deciding,  and  those  that  should  be  abandoned. 

For  example,  the  right-hand  column  in  Table  3  suggests  that 
increasing  the  fuel  capacity  should  be  pursued  before  redesigning  the 
transmission  and  that  reducing  the  time  at  the  base  should  not  be 
considered  further. 

■  Guidance  for  Alternative  Selection:  The  analysis  specifically 
identifies  which  alternatives  improve  which  subprocesses  and  the 
whole  weapon  system  and  by  how  much.  For  example,  Table  3 
suggests  that  increasing  fuel  capacity  increases  the  Fire  mission 
development  subprocess  most,  and  three  alternatives  significantly 
improve  the  engage  target  subprocess  most. 

■  Justification  of  Analysis  of  Alternatives  Decisions:  When  used  with 
the  simulation  model,  the  KVA  productivity  ratios  can  help  explain  and 
justify  Analysis  of  Alternatives  decisions  by  providing  a  means  of 
describing  how  each  alternative  impacts  operations,  subprocesses, 
and  performance.  For  example,  in  the  UAV  case  above,  increasing 
power  plant  size  increases  the  payload,  which  increases  the 
payload/lethal  payload  ratio,  which  increases  the  probability  of 
destruction  if  hit  (p(kill)),  which  decreases  the  return  flow  “Incomplete 
kill  rate”  from  the  Battlefield  Assessment  Backlog  to  the  Target 
engagement  backlog  (Figure  3).  This  reduces  the  average  number  of 
times  that  a  target  must  be  engaged  in  order  to  be  destroyed,  thereby 
improving  the  productivity  of  the  engage  target  subprocess. 

■  Guidance  for  Further  Investigation:  In  addition  to  suggesting  better 
and  worse  alternatives  to  pursue,  an  integrated  system  dynamics/KVA 
model  can  provide  guidance  for  further  investigation  of  alternatives  by 
indicating  which  subprocesses  each  alternative  improves.  For 
example,  Table  3  indicates  that  the  reason  that  increasing  fuel  capacity 
improves  performance  is  that  it  improves  the  Fire  mission  development 
subprocess.  The  model  (Figure  3)  indicates  that  this  occurs  by 
increasing  the  vehicle  range,  which  reduces  the  likelihood  of  losing  a 
target  if  it  is  missed  with  ordinance.  Acquisition  program  managers  can 
use  this  information  to  focus  further  investigation  and  development  of 
this  alternative  on  fire  mission  development  to  assure  that  these 
improvements  in  the  specific  operations  identified  with  the  model  are 
realized  during  the  alternative’s  development. 

It  is  important  to  note  that  neither  a  system  dynamics  model  nor  a  KVA 
analysis  of  this  system  alone  can  reasonably  produce  these  results.  Only  by 
integrating  system  dynamics  and  the  KVA  approach  are  the  benefits  above 
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available.  Based  on  the  modeling  and  assessment  above,  we  conclude  that 
integrated  system  dynamics/KVA  models  can  significantly  improve  the  Analysis  of 
Alternatives  and,  thereby,  acquisition. 

Implications  for  Practice 

The  current  work  indicates  that  acquisition  can  be  improved  by  using 
integrated  system  dynamics/KVA  models  in  the  Analysis  of  Alternatives.  The 
rigorous  development  and  use  of  integrated  system  dynamics/KVA  models  can  have 
important  implications  for  acquisition  practice,  including  the  following: 

■  The  number  of  alternatives  that  can  be  analyzed  with  KVA  can  be 
increased  due  to  the  relative  ease  of  reflecting  alternatives  in  the 
operations  simulation  model  compared  to  manually  developing 
forecasts  for  use  in  KVA  analysis.  This  increases  the  likelihood  of 
identifying  and  selecting  the  optimal  alternative. 

■  Justifications  of  AoA  decisions  can  become  stronger  due  to  program 
managers  having  the  ability  to  causally  trace  from  specific  alternatives 
through  their  impacts  on  specific  subprocesses  and  operations  to 
performance. 

■  Justifications  of  an  AoA  decisions  can  become  more  robust  because 
they  can  reflect  an  analysis  of  a  wider  range  of  alternatives  and  more 
alternatives. 

■  Results  of  AoA  can  become  more  consistent  through  the  use  of  a 
single,  integrated  model  of  system  operations  and  KVA  metrics  instead 
of  separate  operations  and  value-added  models. 

■  System  dynamics/KVA  models  may  be  used  to  baseline  product 
performance  during  the  acquisition  process.  Performance  of  the 
product  can  be  tracked  over  time  and  used  to  improve  the  model  and, 
thereby,  performance  forecasts  and  AoA  later  in  acquisition. 

■  Program  management  will  select  better  alternatives  due  to  the 

implications  for  practice  listed  in  the  bullets  immediately  above.  This 
will  generate  more  effective  and  potentially  cheaper  materiel  solutions. 

In  addition,  improving  the  AoA  and  acquisition  through  integrated  system 
dynamics/KVA  models  can  improve  CONOPS.  The  Javelin  case  study  provides  a 
vivid  example  of  the  ability  of  acquisition  in  general  and  improved  AoA  to  impact 
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tactics  and  strategy.  Upon  receipt  and  use  of  Javelin,  operators  expressed  surprise 
that  its  range  was  twice  that  of  the  weapon  it  replaced.  That  increased  range 
initiated  improvements  to  tactics,  techniques,  and  procedures  (ttp)  such  as  the  use 
of  Javelin  to  detonate  improvised  explosive  devices.  This,  in  turn,  could  generate 
changes  to  strategies.  Accurate  forecasts  of  product  subprocess  performance  (e.g., 
accuracy  at  longer  range)  could  be  used  to  plan  CONOPS  improvements  before 
product  delivery. 

It  is  important  to  note  that  the  purpose  of  the  simulations  of  operations 
developed  and  illustrated  in  the  current  work  was  to  capture  the  relative  benefits  and 
costs  of  different  materiel  alternatives,  not  to  simulate  the  impacts  of  operations  on 
opposing  forces.  The  usefulness  of  models  can  only  be  judged  in  relation  to  the 
specific  purpose  for  which  they  are  built  (Sterman,  2000).  Therefore,  because  the 
purpose  of  integrated  system  dynamics/KVA  models  is  to  improve  AoA,  those 
models  should  be  developed,  assessed,  and  used  separately  from  force-on-force 
and  other  simulations  of  operations  developed  for  other  purposes. 

Future  Work 

The  current  proof-of-concept  work  has  demonstrated  the  potential  of 
integrated  system  dynamics/KVA  models  to  improve  an  Analysis  of  Alternatives  and 
acquisition.  Additional  research  can  extend  this  work  toward  implementation  and 
expanded  application.  Opportunities  include  the  following: 

■  Modeling  a  specific  acquisition  program  in  support  of  its  Analysis  of 
Alternatives  process  can  develop  and  demonstrate  the  capability  of 
operationalizing  the  approach  tested  here. 

■  If  important  uncertainties  in  system  operations  are  incorporated  into 
the  system  dynamics  model,  then  it  can  be  used  to  generate 
distributions  of  KVA  productivities.  These  can  be  used  to  estimate  the 
volatilities  used  in  real  options  analysis,  which  has  been  demonstrated 
to  be  useful  in  DoD  acquisition. 

■  The  application  of  integrated  system  dynamics/KVA  modeling  to  DoD 
product  lifecycle  management  can  be  investigated  by  using  the  model 
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to  generate  forecasts  of  performance  and  KVA  ratios  during 
acquisition,  comparing  those  forecasts  with  actual  operations,  and 
using  the  results  to  improve  the  model  fidelity  with  the  system.  The 
improved  model  can  then  be  used  to  analyze  proposed  changes  or 
replacement  of  the  system  throughout  its  lifecycle. 


The  Analysis  of  Alternatives  is  a  particularly  challenging  part  of  DoD 
acquisition.  Integrating  system  dynamics  modeling  and  the  Knowldege  Value  Added 
approach  has  been  shown  to  be  capable  of  improving  that  analysis  and,  thereby, 
alternative  selection.  Adapting  this  approach  can  significantly  change  and  improve 
DoD  acquisition  practice. 
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Appendix  A.  The  System  Dynamics  Model 


Max  asset  range  available=RANDOM  UNIFORM(0,  1 , 9876)  ~  Dmnl 

Asset  available  range-'Asset  max.  range"*Max  asset  range  available 
NauticalMiles 

"Avail  Range:  Traget  distance  from  base  ratio"=(Asset  available  range/Target 
distance  fr  base)+0. 01  ~  Dmnl 

Confirm  kill  rate=Battlefield  assessment  rate*Fraction  not  confirmed  kill 
target/Minute 

Change  Battlefliedl  assessment  capacity  applied=Max(-0.5*Battlefield  assessment 
capacity  applied, Avg  Battlefield  assessment  delay*Sesnativity  of  Battlefield 
assessment  capability  to  Avg  Battlefield  assessment  delay) 

(targets/Minute)/Minute 

Fraction  not  confirmed  kill=Min(1,"p(incomplete  kill)"+"p(lose  target  after 
miss)"+"p(missed  but  not  lost)")  ~ 

"Dash  speed:  Target  speed  ratio"=(Asset  dash  speed/Target  speed)+1  ~  Dmnl 

"p(not  lose  target)'- "Range-p(not  lose)  function"("Avail  Range:  Traget  distance  from 

base  ratio")  ~  Dmnl 

"Range-p(not  lose)  function"(  [(0,0)- 

(2,1)], (0,0), (0.740061, 0.0482456), (0.91 1315,0. 127193), (1.34557, 0.912281), (1.5596 
3, 0.951754), (1.85933, 0.951754))  ~  | 


"Payload:  Lethal  payload  ratio"=(Asset  payload/Lethal  payload)+0.01  ~  Dmnl 

"Payload  -  p(kill)  function"([(0,0)-(1 0, 1 )], (0,0), (1,1), (10,1))  ~  | 

"Speed-p(hit)  function"([(0,0)- 

(4,1)], (0,0), (0.40367, 0.0219298), (0.648318, 0.0482456), (0.892966, 0.0877193), (1.24 
771 , 0.447368), (1 .57798, 0.745614), (1 .89602,0.868421 ), (2.28746, 0.903509), (2.5932 
7, 0.903509), (2.99694, 0.903509)) 
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Dmnl 


Lethal  payload=RANDOM  UNIFORM(  400,  1000,  1234)  ~  lbs 

"p(hit  target)"="Speed-p(hit)  function"("Dash  speed:  Target  speed  ratio")  ~ 

"p(kill  target  if  hit)"=  "Paylod  -  p(kill)  function"("Payload:  Lethal  payload  ratio") 

Dmnl 

Target  speed=  RANDOM  UNIFORM(50,  250,  1234)  -  knots 

Target  distance  fr  base=RANDOM  UNIFORM(400,  1100,  1234) 

NauticalMiles 

Mission  fraction  sent  to  asset=Min(1  ,Max(0, Reference  Mission  fraction  sent  to 
asset+(Targets  in  fire  mission  development*Sensativity  of  Mission  Fraction  Sent  to 
Asset  to  Backlog)))  ~  Dmnl 

"Fire  support  -  missions  not  assigned  to  asset"=Fire  support  coordination  completion 
rate-Fire  support  coordination  to  asset  rate  ~  target/Minute 

Targets  in  battlefield  assessment  backlog=  INTEG  (  Engage  target  rate-Confirm 
kill  rate-incomplete  kill  rate-Miss  target  but  have  not  lost  target  rate-Lose  target  after 
miss  rate,0)  ~  targets 

Total  confirmed  target  kills=  INTEG  (Confirm  kill  rate,0)  -targets 

Avg  Battlefield  assessment  delay=ZIDZ(Targets  in  battlefield  assessment 
backlog, Confirm  kill  rate)-  Minute 

Targets  in  fire  support  coordination=  INTEG  (Acquire  target  completion  rate-Fire 
support  coordination  to  asset  rate-"Fire  support  -  missions  not  assigned  to  asset", 0) 
targets 

Total  targets  not  assigned  to  asset=  INTEG  ("Fire  support  -  missions  not  assigned  to 
asset", 0)  -  targets 

Asset  payload=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'c22')  -  lbs 
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Target  ID  rate=IF  THEN  ELSE(Time<480,  5,  0)~  targets/Minute 

"Asset  max.  range"=GET  XLS  CONSTANTS('lnput.xls7lnput\'c24') 

NauticalMiles 

Asset  dash  speed=GET  XLS  CONSTANTS('lnput.xls7lnput7c23')  ~  knots 

Cumm  Project  Learning  Time  earned=  INTEG  (Earn  Project  Learning  Time,0) 
LThours 

Cumm  Project  Touch  time  spent=  INTEG  (Spend  Project  Touch  Time,0) 

TThours 

Fire  support  coordination  completion  rate=  Min(Fire  support  coordination 
capacity  applied, Targets  in  fire  support  coordination/Avg  Fire  support  coordination 
duration)  ~  targets/Minute 

Fire  support  coordination  to  asset  rate=Fire  support  coordination  completion 
rate*Mission  fraction  sent  to  asset  ~  targets/Minute 

Reference  Mission  fraction  sent  to  asset=GET  XLS 
CONSTANTS('lnput.xls';input','c16')  ~  Dmnl 

Spend  engagement  touch  time=Engage  target  rate*Avg  Engagement  duration*"unit 
Touch  Time  hours  spent  per  duration  minute  (targets)"  ~  TThours/minutes 

Spend  fire  mission  development  touch  time=  Fire  mission  completion  rate*Avg 
Fire  mission  development  duration*"unit  Touch  Time  hours  spent  per  duration 
minute  (targets)"  ~  TThours/minutes 

Earn  Project  Learning  Time="Earn  LT  -  acquire  targets"+"Earn  LT  -  Battlefield 
assessment"+"Earn  LT  -  Engage  targets"+"Earn  LT  -  Fire  mission 
development"+"Earn  LT  -  Fire  support  coordination"+"Earn  LT  -  Move  assets" 
LThours/Minute 

Targets  in  fire  mission  development=  INTEG  (+Fire  support  coordination  to  asset 
rate+Miss  target  but  have  not  lost  target  rate-Fire  mission  completion  rate,0) 
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targets 


Sensativity  of  Mission  Fraction  Sent  to  Asset  to  Backlog=-0.05  ~  1 /targets 

Spend  Project  Touch  Time=Spend  battle  assessment  touch  time+Spend 

engagement  touch  time+Spend  fire  mission  development  touch 
time+Spend  fire  support  coordination  touch  time+Spend  target 
acquisition  touch  time+Total  spend  move  asset  touch  time 
TThours/minutes 

"Earn  revenue  -  Fire  mission  development'^  Fire  mission  completion  rate*"Unit 
revenue  -  Fire  mission  development"  ~  kdollars/Minute 

"Earn  LT  -  Move  assets"=Total  move  asset  rate*Move  asset  unit  learning  time*"Avg 
no.  of  targets  that  each  asset  can  engage  at  once"  ~  LThours/Minute 

Fire  mission  completion  rate=Min(Fire  mission  development  capacity 
applied, Targets  in  fire  mission  development/Avg  Fire  mission  development  duration) 
targets/Minute 

Project  productivity=XIDZ(Cumm  Project  Learning  Time  earned, (Cumm  Project 
Touch  time  spent*Relative  value  of  LT  compared  to  TT),  1 )  ~  Dmnl 

Engage  target  rate=Min(Target  engagement  backlog, Targets  assets  on  station  can 
engage)/Avg  Engagement  duration-  targets/Minute 

Assets  in  route  to  Base=  INTEG  (Assets  leave  station  for  base  rate-Assets  arriving 

at  base  rate,0)  -  Assets 

Assets  arrive  at  station  rate=Assets  in  route  to  Station/Avg  travel  time  to  or  from 
station  -  Assets/Minute 

Assets  arriving  at  base  rate=Assets  in  route  to  Base/Avg  travel  time  to  or  from 
station  -  Assets/Minute 

Assets  at  Base=  INTEG  (Assets  arriving  at  base  rate-Assets  leaving  base  for  station 
rate, Number  of  assets  applied)  -  Assets 
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Avg  travel  time  to  or  from  station=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'm7') 
Minute 

Assets  leave  station  for  base  rate=Assets  on  Station/Avg  time  on  station 
Assets/Minute 

Assets  leaving  base  for  station  rate=Assets  at  Base/Avg  time  at  base 
Assets/Minute 

Avg  time  at  base=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'ml  1')  ~  Minute 
Assumed  to  be  the  average  time  from  arrival  of  the  asset  at  the  base 

that  \is  required  to  refuel  and  reload  asset  and  have  it  depart  from 

base. 

Incomplete  kill  rate=Battlefield  assessment  rate*"p(incomplete  kill)" 
targets/Minute 

"Spend  move  asset  touch  time-base  to  station"=Assets  leaving  base  for  station 
rate*Avg  travel  time  to  or  from  station*"unit  Touch  Time  hours  spent  per  duration 
minute  (assets)"  ~  TThours/minutes 

"Spend  move  asset  touch  time-station  to  base"=Assets  leave  station  for  base 
rate*Avg  travel  time  to  or  from  station*"unit  Touch  Time  hours  spent  per  duration 
minute  (assets)"  TThours/minutes 

Miss  target  but  have  not  lost  target  rate=Battlefield  assessment  rate*"p(missed  but 
not  lost)" 

targets/Minute 

Total  move  asset  rate=Assets  leave  station  for  base  rate+Assets  arriving  at  base 
rate  ~  Assets/Minute 

Lose  target  after  miss  rate=  Battlefield  assessment  rate*"p(lose  target  after 
miss)"~targets/Minute 
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Targets  being  acquired=  INTEG  (+Target  ID  rate+Lose  target  after  miss  rate-Acquire 
target  completion  rate,0)  ~  targets 

Targets  assets  on  station  can  engage=INTEGER(Assets  on  Station*"Avg  no.  of 
targets  that  each  asset  can  engage  at  once")  ~  targets 

"p(missed  but  not  lost)"=(1-"p(hit  target)")*(1-"p(not  lose  target)")  ~  Dmnl 

"p(incomplete  kill)"="p(hit  target)"*(1-"p(kill  target  if  hit)")  ~  Dmnl 

"p(lose  target  after  miss)"=(1-"p(hit  target)")*(1-"p(not  lose  target)")  ~  Dmnl 

Battlefield  assessment  capacity  applied=  INTEG  (+Change  Battlefliedl  assessment 
capacity  applied, Reference  Battlefield  assessment  capability)  ~  target/Minute 

Battlefield  assessment  rate=Min(Battlefield  assessment  capacity  applied, Targets  in 
battlefield  assessment  backlog/Avg  Battle  assessment  duration)  ~  target/Minute 

"Cumm  Learning  Time  -  acquire  targets'^  INTEG  ("Earn  LT  -  acquire  targets", 0) 
LThours 

Spend  fire  support  coordination  touch  time=(Fire  support  coordination  to  asset 
rate*Mission  fraction  sent  to  asset)*Avg  Fire  support  coordination  duration*"unit 
Touch  Time  hours  spent  per  duration  minute  (targets)"  ~  TThours/minutes 

"Cumm  Learning  Time  -  engage  targets'^  INTEG  ("Earn  LT  -  Engage  targets", 0) 
LThours 

"Cumm  Learning  Time  -  Fire  mission  development'^  INTEG  ("Earn  LT  -  Fire  mission 
development",  0)  ~  LThours 

Spend  target  acquisition  touch  time=(Acquire  target  completion  rate*Mission  fraction 
sent  to  asset)*Avg  Target  acquisition  duration*"unit  Touch  Time  hours  spent  per 
duration  minute  (targets)"  ~  TThours/minutes 

"Cumm  Learning  Time  -  Move  assets"=  INTEG  ("Earn  LT  -  Move  assets", 0) 
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LThours 


Sensitivity  of  Battlefield  assessment  capability  to  Avg  Battlefield  assessment 
delay=0.01  ~  ((targets/Minute)/Minute)/Minute 

"Earn  LT  -  acquire  targets"=(Acquire  target  completion  rate*Mission  fraction  sent  to 
asset)*Acquire  target  unit  learning  time  ~  LThours/Minute 

"Earn  LT  -  Battlefield  assessment"=Battlefield  assessment  rate*Battle  assessment 

unit  learning  time  ~  LThours/Minute 

"Earn  LT  -  Engage  targets'-Engage  target  rate*Target  engagement  unit  learning 
time  ~  LThours/Minute 

"Productivity  -  Move  assets"=ZIDZ("Cumm  Learning  Time  -  Move  assets", ("Cumm 
Touch  Time  -  Move  assets"*Relative  value  of  LT  compared  to  TT))  ~  Dmnl 

"Earn  LT  -  Fire  mission  development"=Fire  mission  completion  rate*Fire  mission  unit 
learning  time  ~  LThours/Minute 

"Earn  LT  -  Fire  support  coordination"=(Fire  support  coordination  to  asset 
rate*Mission  fraction  sent  to  asset)*Fire  support  coordination  unit  learning  time 
LThours/Minute 

"Cumm  Learning  Time  -  Battlefield  assessment'^  INTEG  ("Earn  LT  -  Battlefield 
assessment", 0)  ~  LThours 

Reference  Battlefield  assessment  capability=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'g5')  ~  targets/Minute 

Relative  value  of  LT  compared  to  TT=1  ~  LThours/TThours 

"Productivity  -  Engage  targets"=ZIDZ("Cumm  Learning  Time  -  engage 
targets", ("Cumm  Touch  Time  -  Engage  targets"*Relative  value  of  LT  compared  to 
TT))  ~  Dmnl 
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"Productivity  -  Acquire  targets"=ZIDZ("Cumm  Learning  Time  -  acquire 

targets", ("Cumm  Touch  Time  -  Target  acquisition"*Relative  value  of  LT  compared  to 

TT))  ~  Dmnl 

"Cumm  Learning  Time  -  Fire  support  coordination'  -  INTEG  ("Earn  LT  -  Fire  support 
coordination", 0)  ~  LThours 

"Productivity  -  Fire  mission  development"=ZIDZ("Cumm  Learning  Time  -  Fire 
mission  development", ("Cumm  Touch  time  -  Fire  mission  development"*Relative 
value  of  LT  compared  to  TT))  ~  Dmnl 

"Productivity  -  Fire  support  coordination"=ZIDZ("Cumm  Learning  Time  -  Fire  support 
coordination", ("Cumm  Touch  time  -  Fire  support  coordination"*Relative  value  of  LT 
compared  to  TT))  ~  Dmnl 

"Productivity  -  Battlefield  assessment"=ZIDZ("Cumm  Learning  Time  -  Battlefield 
assessment", ("Cumm  Touch  Time  -  Battlefield  assessment"*Relative  value  of  LT 
compared  to  TT))  ~  Dmnl 

"Cumm  work  cost  -  Battlefield  assessment"="Cumm  Touch  Time  -  Battlefield 
assessment"*"Hourly  work  cost  -  Battlefield  assessment"  ~  kdollars 

"Cumm  work  cost  -  Engage  target"="Cumm  Touch  Time  -  Engage  targets"*"Hourly 
work  cost  -  engage  target"  ~  kdollars 

"Cumm  work  cost  -  fire  mission  development"="Cumm  Touch  time  -  Fire  mission 
development"*"Hourly  work  cost  -  Fire  mission  development"  ~  kdollars 

"Cumm  work  cost  -  Fire  support  coordination"="Cumm  Touch  time  -  Fire  support 
coordination"*"Hourly  work  cost  -  Fire  support  coordination"  ~  kdollars 

"Cumm  work  cost  -  Move  launchers"="Cumm  Touch  Time  -  Move  assets"*"Hourly 
work  cost  -  Move  launchers"  ~  kdollars 
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Cumm  Fire  mission  development  $  productivity=ZIDZ("Cumm  revenue  -  Fire  mission 
development", "Cumm  work  cost  -  fire  mission  development")  ~  Dmnl 

Cumm  Fire  support  coordination  $  productivity=ZIDZ("Cumm  revenue  -  Fire  support 
coordination", "Cumm  work  cost  -  Fire  support  coordination")  ~  Dmnl 

"Earn  revenue  -  engage  target"=Engage  target  rate*"Unit  revenue  -  engage  target" 
kdollars/Minute 

"Hourly  work  cost  -  engage  target"=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'i8') 
kdollars/TThour 

"Hourly  work  cost  -  Fire  mission  development"=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'i6')  ~  kdollars/TThour 

"Earn  revenue  -  Move  launchers'-Total  move  asset  rate*("Unit  revenue  -  Move 
launchers"*"Avg  no.  of  targets  that  each  asset  can  engage  at  once") 


kdollars/Minute 


"Hourly  work  cost  -  Battlefield  assessment"=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'i9')  ~  kdollars/TThour 

Cumm  Battle  assessment  $  productivity=ZIDZ("Cumm  revenue  -  Battlefield 
assessment", "Cumm  work  cost  -  Battlefield  assessment")  ~  Dmnl 

"Cumm  work  cost  -  Acquire  target"="Cumm  Touch  Time  -  Target  acquisition"*"Hourly 
work  cost  -  Acquire  target"  ~  kdollars 

Cumm  target  engagement  $  productivity=ZIDZ("Cumm  revenue  -  engage 
target", "Cumm  work  cost  -  Engage  target")  ~  Dmnl 

Cumm  Acquire  target  $  productivity=ZIDZ("Cumm  revenue  -  Acquire  target", "Cumm 
work  cost  -  Acquire  target")  ~  Dmnl 

"Hourly  work  cost  -  Fire  support  coordination"=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'i5')  ~  kdollars/TThour 
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Cumm  move  launcher  $  prodictivity=ZIDZ("Cumm  revenue  -  Move 
launchers", "Cumm  work  cost  -  Move  launchers")  ~  Dmnl 

"Hourly  work  cost  -  Move  launchers"=GET  XLS  CONSTANTS('lnput.xls';input','i7') 
kdollars/TThour 

"Cumm  revenue  -  Acquire  target"=  INTEG  (  "Earn  revenue  -  Acquire  target", 0) 
kdollars 

"Cumm  revenue  -  Fire  mission  development'  -  INTEG  ("Earn  revenue  -  Fire  mission 
development",  0)  ~  kdollars 

"Cumm  revenue  -  Fire  support  coordination'  -  INTEG  ("Earn  revenue  -  Fire  support 
coordination", 0)  ~  kdollars 

"Cumm  revenue  -  Move  launchers'-  INTEG  ("Earn  revenue  -  Move  launchers", 0) 
kdollars 

"Cumm  revenue  -  Battlefield  assessment'^  INTEG  ("Earn  revenue  -  Battlefield 
assessment", 0)  ~  kdollars 

"Earn  revenue  -  Acquire  target"=Acquire  target  completion  rate*"Unit  revenue- 
Acquire  targets"  ~  kdollars/Minute 

"Earn  revenue  -  Battlefield  assessment'-Battlefield  assessment  rate*"Unit  revenue  - 
battlefield  assessment"  ~  kdollars/Minute 

"Earn  revenue  -  Fire  support  coordination"=Fire  support  coordination  to  asset 
rate*"Unit  revenue  -  Fire  support  coordination"  ~  kdollars/Minute 

Total  process  learning  time=Acquire  target  unit  learning  time+Battle  assessment  unit 
learning  time+Target  engagement  unit  learning  time+Fire  mission  unit  learning 
time+Fire  support  coordination  unit  learning  time+Move  asset  unit  learning  time 
LThours/target 
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"Cumm  revenue  -  engage  target"=  INTEG  ("Earn  revenue  -  engage  target", 0)~ 
kdollars 

"Revenue  fraction  -  Engage  target"=Target  engagement  unit  learning  time/Total 


process  learning  time 


Dmnl 


"Hourly  work  cost  -  Acquire  target"=  GET  XLS 
CONSTANTSCInput.xIs', 'Input', 'i4')  ~  kdollars/TThour 

"Revenue  fraction  -  Battlefield  assessment'-  Battle  assessment  unit  learning 
time/Total  process  learning  time  ~  Dmnl 

"Revenue  fraction  -  Fire  mission  development"=Fire  mission  unit  learning  time/Total 
process  learning  time  ~  Dmnl 

"Revenue  fraction  -  Fire  support  coordination"=Fire  support  coordination  unit 
learning  time/Total  process  learning  time  ~  Dmnl 

"Revenue  fraction  -  move  launchers"=Move  asset  unit  learning  time/Total  process 
learning  time  ~  Dmnl 

"Revenue  fraction  -  Acquire  target"=  Acquire  target  unit  learning  time/Total 
process  learning  time  ~  Dmnl 

"Unit  revenue-  Acquire  targets"="Revenue  fraction  -  Acquire  target"*Unit  revenue  of 
process  ~  kdol  lars/target 

"Unit  revenue  -  engage  target"="Revenue  fraction  -  Engage  target"*Unit  revenue  of 
process  ~  kdol  lars/target 

"Unit  revenue  -  Move  launchers"="Revenue  fraction  -  move  launchers"*Unit  revenue 
of  process  ~  kdol  lars/target 

"Unit  revenue  -  battlefield  assessment"="Revenue  fraction  -  Battlefield 
assessment"*Unit  revenue  of  process  ~  kdollars/target 
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"Unit  revenue  -  Fire  support  coordination"="Revenue  fraction  -  Fire  support 
coordination"*Unit  revenue  of  process  ~  kdollars/target 

Unit  revenue  of  process=1e+006  ~  kdollars/target 

"Unit  revenue  -  Fire  mission  development"="Revenue  fraction  -  Fire  mission 
development"*Unit  revenue  of  process  ~  kdollars/target 

Number  of  assets  applied=GET  XLS  CONSTANTS('lnput.xls';input','G7') 

Assets 

Spend  battle  assessment  touch  time=Battlefield  assessment  rate*Avg  Battle 
assessment  duration*"unit  Touch  Time  hours  spent  per  duration  minute  (targets)" 
TThours/minutes 

"unit  Touch  Time  hours  spent  per  duration  minute  (targets)'-0. 01 76667 
TThours/(minutes*target)  ~  1  60th  of  an  hour 

Total  spend  move  asset  touch  time="Spend  move  asset  touch  time-station  to 
base"+"Spend  move  asset  touch  time-base  to  station"  ~ 

TThours/minutes 

"unit  Touch  Time  hours  spent  per  duration  minute  (assets)'-0. 01 76667 
TThours/(asset*minutes)  ~  1  60th  of  an  hour 

Target  engagement  unit  learning  time=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'f8') 
LThours/target 

"Avg  no.  of  targets  that  each  asset  can  engage  at  once"=5  ~  targets/asset 

Acquire  target  completion  rate=  Min  (Target  acquisition  capacity  applied, Targets 
being  acquired/Avg  Target  acquisition  duration)  ~  targets/Minute 

Acquire  target  unit  learning  time=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'f4') 
LThours/target 

Battle  assessment  unit  learning  time=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'f9') 
LThours/target 
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Fire  mission  unit  learning  time=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'f6') 

LTh  ours/target 

Target  engagement  backlog=  INTEG  (+Fire  mission  completion  rate+lncomplete  kill 

rate-Engage  target  rate.O)  ~  targets 

Avg  time  on  station=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'm5')  ~  Minute 

Avg  Battle  assessment  duration=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'e9') 
Minute 

"Cumm  Touch  Time  -  Battlefield  assessment'  -  INTEG  (Spend  battle  assessment 
touch  time,0)  ~  TThours 

Avg  Engagement  duration=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'e8') 

Minute 

"Cumm  Touch  Time  -  Engage  targets'^  INTEG  (Spend  engagement  touch  time,0) 
TThours 

Fire  mission  development  capacity  applied=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'g6')  ~  targets/Minute 

Avg  Fire  mission  development  duration=GET  XLS 
CONSTANTSCInput.xls',’ 'Input', 'e6')  ~  Minute 

"Cumm  Touch  time  -  Fire  mission  development'^  INTEG  (Spend  fire  mission 
development  touch  time,0)  ~  TThours 

Fire  support  coordination  capacity  applied=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'g5')  ~  targets/Minute 

Avg  Fire  support  coordination  duration=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'e5')  ~  Minute 

"Cumm  Touch  time  -  Fire  support  coordination"=  INTEG  (Spend  fire  support 
coordination  touch  time.O)  ~  TThours 

Fire  support  coordination  unit  learning  time=GET  XLS 
CONSTANTSCInput.xls', 'Input', 'f5')  ~  LThours/target 
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Assets  on  Station=  INTEG  (Assets  arrive  at  station  rate-Assets  leave  station  for 
base  rate,0)  ~  Assets 

Assets  in  route  to  Station=  INTEG  (+Assets  leaving  base  for  station  rate-Assets 
arrive  at  station  rate.O)  ~  Assets 

Move  asset  unit  learning  time=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'f7') 
LThours/target 

"Cumm  Touch  Time  -  Move  assets'-  INTEG  (Total  spend  move  asset  touch  time,  0) 
TThours 

Target  acquisition  capacity  applied=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'G4') 
targets/Minute 

Avg  Target  acquisition  duration=GET  XLS  CONSTANTS('lnput.xls', 'Input', 'e4') 
Minute 

"Cumm  Touch  Time  -  Target  acquisition'^  INTEG  (Spend  target  acquisition  touch 
time,0)  ~  TThours 

Simulation  Control  Parameters 

FINAL  TIME  =  900  Minute  ~  The  final  time  for  the  simulation. 

INITIAL  TIME  =  0~  Minute  ~  The  initial  time  for  the  simulation. 

SAVEPER  =  0.25-  Minute  [0,?]~The  frequency  with  which  output  is  stored. 

TIME  STEP  =0.125  Minute  [0,?]~  The  time  step  for  the  simulation. 
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2003  -2010  Sponsored  Research  Topics 


Acquisition  Management 

■  Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

■  BCA:  Contractor  vs.  Organic  Growth 

■  Defense  Industry  Consolidation 

■  EU-US  Defense  Industrial  Relationships 

■  Knowledge  Value  Added  (KVA)  +  Real  Options  (RO)  Applied  to 
Shipyard  Planning  Processes 

■  Managing  the  Services  Supply  Chain 

■  MOSA  Contracting  Implications 

■  Portfolio  Optimization  via  KVA  +  RO 

■  Private  Military  Sector 

■  Software  Requirements  for  OA 

■  Spiral  Development 

■  Strategy  for  Defense  Acquisition  Research 

■  The  Software,  Hardware  Asset  Reuse  Enterprise  (SHARE)  repository 

Contract  Management 

■  Commodity  Sourcing  Strategies 

■  Contracting  Government  Procurement  Functions 

■  Contractors  in  21st-century  Combat  Zone 

■  Joint  Contingency  Contracting 

■  Model  for  Optimizing  Contingency  Contracting,  Planning  and  Execution 

■  Navy  Contract  Writing  Guide 

■  Past  Performance  in  Source  Selection 

■  Strategic  Contingency  Contracting 

■  Transforming  DoD  Contract  Closeout 

■  USAF  Energy  Savings  Performance  Contracts 

■  USAF  IT  Commodity  Council 

■  USMC  Contingency  Contracting 
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Financial  Management 

■  Acquisitions  via  Leasing:  MPS  case 

■  Budget  Scoring 

■  Budgeting  for  Capabilities-based  Planning 

■  Capital  Budgeting  for  the  DoD 

■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Financing  DoD  Budget  via  PPPs 

■  Lessons  from  Private  Sector  Capital  Budgeting  for  DoD  Acquisition 
Budgeting  Reform 

■  PPPs  and  Government  Financing 

■  ROI  of  Information  Warfare  Systems 

■  Special  Termination  Liability  in  MDAPs 

■  Strategic  Sourcing 

■  Transaction  Cost  Economics  (TCE)  to  Improve  Cost  Estimates 

Human  Resources 

■  Indefinite  Reenlistment 

■  Individual  Augmentation 

■  Learning  Management  Systems 

■  Moral  Conduct  Waivers  and  First-tern  Attrition 

■  Retention 

■  The  Navy’s  Selective  Reenlistment  Bonus  (SRB)  Management  System 

■  Tuition  Assistance 

Logistics  Management 

Analysis  of  LAV  Depot  Maintenance 
Army  LOG  MOD 
ASDS  Product  Support  Analysis 
Cold-chain  Logistics 

Contractors  Supporting  Military  Operations 
Diffusion/Variability  on  Vendor  Performance  Evaluation 
Evolutionary  Acquisition 

Lean  Six  Sigma  to  Reduce  Costs  and  Improve  Readiness 
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■  Naval  Aviation  Maintenance  and  Process  Improvement  (2) 

■  Optimizing  CIWS  Lifecycle  Support  (LCS) 

■  Outsourcing  the  Pearl  Harbor  MK-48  Intermediate  Maintenance 
Activity 

■  Pallet  Management  System 

■  PBL  (4) 

■  Privatization-NOSL/NAWCI 

-  RFID  (6) 

■  Risk  Analysis  for  Performance-based  Logistics 

■  R-TOC  AEGIS  Microwave  Power  Tubes 

■  Sense-and-Respond  Logistics  Network 

■  Strategic  Sourcing 

Program  Management 

■  Building  Collaborative  Capacity 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Collaborative  IT  Tools  Leveraging  Competence 

■  Contractor  vs.  Organic  Support 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

-  KVA  Applied  to  AEGIS  and  SSDS 

■  Managing  the  Service  Supply  Chain 

■  Measuring  Uncertainty  in  Earned  Value 

■  Organizational  Modeling  and  Simulation 

■  Public-Private  Partnership 

■  Terminating  Your  Own  Program 
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